Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: Add External Source (and External Event) Attribute Validation #116

Open
wants to merge 11 commits into
base: develop
Choose a base branch
from

Conversation

pranav-super
Copy link

@pranav-super pranav-super commented Nov 18, 2024

This pull request adds external source/event attribute validation to Aerie.

This means that when an external source is uploaded to AERIE, via endpoints like the CLI or the UI, it is forwarded here, has its formatting checked against a schema, and additionally has the attributes of the source and each of its events checked for validity, before submitting it to the database.

Submission of event/source type schemas for the purpose of later validation is also handled here - the gateway simply checks that they are valid JSON Schema.

A more thorough explanation of the feature can be found in the AERIE PR, here.

} catch (e) {
logger.error(`POST /uploadExternalEventType: ${(e as Error).message}`);
res.status(500);
res.send(`POST /uploadExternalEventType: ${(e as Error).message}`);

Check warning

Code scanning / CodeQL

Exception text reinterpreted as HTML Medium

Exception text
is reinterpreted as HTML without escaping meta-characters.

Copilot Autofix AI about 13 hours ago

To fix the problem, we need to ensure that any error messages sent in the response are properly sanitized or escaped to prevent XSS attacks. The best way to fix this is to use a library like he (HTML entities) to encode the error messages before sending them in the response.

  1. Install the he library to handle HTML entity encoding.
  2. Import the he library in the file.
  3. Use the he.encode function to encode the error messages before sending them in the response.
Suggested changeset 2
src/packages/external-event/external-event.ts

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/src/packages/external-event/external-event.ts b/src/packages/external-event/external-event.ts
--- a/src/packages/external-event/external-event.ts
+++ b/src/packages/external-event/external-event.ts
@@ -7,2 +7,3 @@
 import { HasuraError } from '../../types/hasura.js';
+import he from 'he';
 
@@ -40,5 +41,6 @@
   } catch (e) {
-    logger.error(`POST /uploadExternalEventType: ${(e as Error).message}`);
+    const errorMessage = he.encode((e as Error).message);
+    logger.error(`POST /uploadExternalEventType: ${errorMessage}`);
     res.status(500);
-    res.send(`POST /uploadExternalEventType: ${(e as Error).message}`);
+    res.send(`POST /uploadExternalEventType: ${errorMessage}`);
     return;
@@ -52,5 +54,6 @@
   } catch (e) {
-    logger.error(`POST /uploadExternalEventType: ${(e as Error).message}`);
+    const errorMessage = he.encode((e as Error).message);
+    logger.error(`POST /uploadExternalEventType: ${errorMessage}`);
     res.status(500);
-    res.send(`POST /uploadExternalEventType: ${(e as Error).message}`);
+    res.send(`POST /uploadExternalEventType: ${errorMessage}`);
     return;
EOF
@@ -7,2 +7,3 @@
import { HasuraError } from '../../types/hasura.js';
import he from 'he';

@@ -40,5 +41,6 @@
} catch (e) {
logger.error(`POST /uploadExternalEventType: ${(e as Error).message}`);
const errorMessage = he.encode((e as Error).message);
logger.error(`POST /uploadExternalEventType: ${errorMessage}`);
res.status(500);
res.send(`POST /uploadExternalEventType: ${(e as Error).message}`);
res.send(`POST /uploadExternalEventType: ${errorMessage}`);
return;
@@ -52,5 +54,6 @@
} catch (e) {
logger.error(`POST /uploadExternalEventType: ${(e as Error).message}`);
const errorMessage = he.encode((e as Error).message);
logger.error(`POST /uploadExternalEventType: ${errorMessage}`);
res.status(500);
res.send(`POST /uploadExternalEventType: ${(e as Error).message}`);
res.send(`POST /uploadExternalEventType: ${errorMessage}`);
return;
package.json
Outside changed files

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/package.json b/package.json
--- a/package.json
+++ b/package.json
@@ -43,3 +43,4 @@
     "swagger-ui-express": "^4.6.3",
-    "winston": "^3.9.0"
+    "winston": "^3.9.0",
+    "he": "^1.2.0"
   },
EOF
@@ -43,3 +43,4 @@
"swagger-ui-express": "^4.6.3",
"winston": "^3.9.0"
"winston": "^3.9.0",
"he": "^1.2.0"
},
This fix introduces these dependencies
Package Version Security advisories
he (npm) 1.2.0 None
Copilot is powered by AI and may make mistakes. Always verify output.
Positive Feedback
Negative Feedback

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

Please select one or more of the options
} catch (e) {
logger.error(`POST /uploadExternalSourceType: ${(e as Error).message}`);
res.status(500);
res.send(`POST /uploadExternalSourceType: ${(e as Error).message}`);

Check warning

Code scanning / CodeQL

Exception text reinterpreted as HTML Medium

Exception text
is reinterpreted as HTML without escaping meta-characters.

Copilot Autofix AI about 13 hours ago

To fix the problem, we need to ensure that any error messages sent back in the response are properly sanitized to prevent XSS attacks. The best way to achieve this is by using a library that provides HTML escaping functionality. One such library is he (HTML entities), which can be used to escape any potentially dangerous characters in the error message.

  1. Install the he library to handle HTML escaping.
  2. Import the he library in the file.
  3. Use the he.escape function to sanitize the error message before sending it in the response.
Suggested changeset 2
src/packages/external-source/external-source.ts

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/src/packages/external-source/external-source.ts b/src/packages/external-source/external-source.ts
--- a/src/packages/external-source/external-source.ts
+++ b/src/packages/external-source/external-source.ts
@@ -1,2 +1,3 @@
 import type { Express, Request, Response } from 'express';
+import he from 'he';
 import type { DerivationGroupInsertInput, ExternalSourceInsertInput, ExternalSourceTypeInsertInput } from '../../types/external-source.js';
@@ -51,3 +52,3 @@
     res.status(500);
-    res.send(`POST /uploadExternalSourceType: ${(e as Error).message}`);
+    res.send(`POST /uploadExternalSourceType: ${he.escape((e as Error).message)}`);
     return;
@@ -65,3 +66,3 @@
     res.status(500);
-    res.send(`POST /uploadExternalSourceType: ${(e as Error).message}`);
+    res.send(`POST /uploadExternalSourceType: ${he.escape((e as Error).message)}`);
     return;
EOF
@@ -1,2 +1,3 @@
import type { Express, Request, Response } from 'express';
import he from 'he';
import type { DerivationGroupInsertInput, ExternalSourceInsertInput, ExternalSourceTypeInsertInput } from '../../types/external-source.js';
@@ -51,3 +52,3 @@
res.status(500);
res.send(`POST /uploadExternalSourceType: ${(e as Error).message}`);
res.send(`POST /uploadExternalSourceType: ${he.escape((e as Error).message)}`);
return;
@@ -65,3 +66,3 @@
res.status(500);
res.send(`POST /uploadExternalSourceType: ${(e as Error).message}`);
res.send(`POST /uploadExternalSourceType: ${he.escape((e as Error).message)}`);
return;
package.json
Outside changed files

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/package.json b/package.json
--- a/package.json
+++ b/package.json
@@ -43,3 +43,4 @@
     "swagger-ui-express": "^4.6.3",
-    "winston": "^3.9.0"
+    "winston": "^3.9.0",
+    "he": "^1.2.0"
   },
EOF
@@ -43,3 +43,4 @@
"swagger-ui-express": "^4.6.3",
"winston": "^3.9.0"
"winston": "^3.9.0",
"he": "^1.2.0"
},
This fix introduces these dependencies
Package Version Security advisories
he (npm) 1.2.0 None
Copilot is powered by AI and may make mistakes. Always verify output.
Positive Feedback
Negative Feedback

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

Please select one or more of the options
} else {
logger.error(`POST /uploadExternalSource: Source's formatting is invalid per basic schema validation:\n${JSON.stringify(compiledExternalSourceSchema.errors)}`);
res.status(500);
res.send(`POST /uploadExternalSource: Source's formatting is invalid per basic schema validation:\n${JSON.stringify(compiledExternalSourceSchema.errors)}`);

Check warning

Code scanning / CodeQL

Exception text reinterpreted as HTML Medium

JSON schema validation error
is reinterpreted as HTML without escaping meta-characters.

Copilot Autofix AI about 13 hours ago

To fix the problem, we need to ensure that any user-controlled input included in the error message is properly escaped before being sent in the response. This can be achieved by using a library like he (HTML entities) to encode the error message, preventing any HTML or JavaScript from being executed.

  1. Install the he library to handle HTML entity encoding.
  2. Import the he library in the file.
  3. Use the he.encode function to encode the error message before sending it in the response.
Suggested changeset 2
src/packages/external-source/external-source.ts

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/src/packages/external-source/external-source.ts b/src/packages/external-source/external-source.ts
--- a/src/packages/external-source/external-source.ts
+++ b/src/packages/external-source/external-source.ts
@@ -9,2 +9,3 @@
 import { HasuraError } from '../../types/hasura.js';
+import he from 'he';
 
@@ -128,5 +129,6 @@
   } else {
-    logger.error(`POST /uploadExternalSource: Source's formatting is invalid per basic schema validation:\n${JSON.stringify(compiledExternalSourceSchema.errors)}`);
+    const errorMessage = `POST /uploadExternalSource: Source's formatting is invalid per basic schema validation:\n${JSON.stringify(compiledExternalSourceSchema.errors)}`;
+    logger.error(errorMessage);
     res.status(500);
-    res.send(`POST /uploadExternalSource: Source's formatting is invalid per basic schema validation:\n${JSON.stringify(compiledExternalSourceSchema.errors)}`);
+    res.send(he.encode(errorMessage));
     return;
EOF
@@ -9,2 +9,3 @@
import { HasuraError } from '../../types/hasura.js';
import he from 'he';

@@ -128,5 +129,6 @@
} else {
logger.error(`POST /uploadExternalSource: Source's formatting is invalid per basic schema validation:\n${JSON.stringify(compiledExternalSourceSchema.errors)}`);
const errorMessage = `POST /uploadExternalSource: Source's formatting is invalid per basic schema validation:\n${JSON.stringify(compiledExternalSourceSchema.errors)}`;
logger.error(errorMessage);
res.status(500);
res.send(`POST /uploadExternalSource: Source's formatting is invalid per basic schema validation:\n${JSON.stringify(compiledExternalSourceSchema.errors)}`);
res.send(he.encode(errorMessage));
return;
package.json
Outside changed files

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/package.json b/package.json
--- a/package.json
+++ b/package.json
@@ -43,3 +43,4 @@
     "swagger-ui-express": "^4.6.3",
-    "winston": "^3.9.0"
+    "winston": "^3.9.0",
+    "he": "^1.2.0"
   },
EOF
@@ -43,3 +43,4 @@
"swagger-ui-express": "^4.6.3",
"winston": "^3.9.0"
"winston": "^3.9.0",
"he": "^1.2.0"
},
This fix introduces these dependencies
Package Version Security advisories
he (npm) 1.2.0 None
Copilot is powered by AI and may make mistakes. Always verify output.
Positive Feedback
Negative Feedback

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

Please select one or more of the options
// source type does not exist!
logger.error(`POST /uploadExternalSource: Source type ${source_type_name} does not exist!`);
res.status(500);
res.send(`POST /uploadExternalSource: Source type ${source_type_name} does not exist!`);

Check failure

Code scanning / CodeQL

Reflected cross-site scripting High

Cross-site scripting vulnerability due to a
user-provided value
.

Copilot Autofix AI about 13 hours ago

To fix the reflected cross-site scripting vulnerability, we need to sanitize the user input before including it in the response. The best way to achieve this is by using a library that provides HTML escaping functionality. In this case, we can use the escape-html library to escape the source_type_name before including it in the response.

  • Import the escape-html library.
  • Use the escape function from the escape-html library to sanitize the source_type_name before including it in the response.
Suggested changeset 2
src/packages/external-source/external-source.ts

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/src/packages/external-source/external-source.ts b/src/packages/external-source/external-source.ts
--- a/src/packages/external-source/external-source.ts
+++ b/src/packages/external-source/external-source.ts
@@ -1,2 +1,3 @@
 import type { Express, Request, Response } from 'express';
+import escape from 'escape-html';
 import type { DerivationGroupInsertInput, ExternalSourceInsertInput, ExternalSourceTypeInsertInput } from '../../types/external-source.js';
@@ -159,5 +160,5 @@
       // source type does not exist!
-      logger.error(`POST /uploadExternalSource: Source type ${source_type_name} does not exist!`);
+      logger.error(`POST /uploadExternalSource: Source type ${escape(source_type_name)} does not exist!`);
       res.status(500);
-      res.send(`POST /uploadExternalSource: Source type ${source_type_name} does not exist!`);
+      res.send(`POST /uploadExternalSource: Source type ${escape(source_type_name)} does not exist!`);
       return;
EOF
@@ -1,2 +1,3 @@
import type { Express, Request, Response } from 'express';
import escape from 'escape-html';
import type { DerivationGroupInsertInput, ExternalSourceInsertInput, ExternalSourceTypeInsertInput } from '../../types/external-source.js';
@@ -159,5 +160,5 @@
// source type does not exist!
logger.error(`POST /uploadExternalSource: Source type ${source_type_name} does not exist!`);
logger.error(`POST /uploadExternalSource: Source type ${escape(source_type_name)} does not exist!`);
res.status(500);
res.send(`POST /uploadExternalSource: Source type ${source_type_name} does not exist!`);
res.send(`POST /uploadExternalSource: Source type ${escape(source_type_name)} does not exist!`);
return;
package.json
Outside changed files

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/package.json b/package.json
--- a/package.json
+++ b/package.json
@@ -43,3 +43,4 @@
     "swagger-ui-express": "^4.6.3",
-    "winston": "^3.9.0"
+    "winston": "^3.9.0",
+    "escape-html": "^1.0.3"
   },
EOF
@@ -43,3 +43,4 @@
"swagger-ui-express": "^4.6.3",
"winston": "^3.9.0"
"winston": "^3.9.0",
"escape-html": "^1.0.3"
},
This fix introduces these dependencies
Package Version Security advisories
escape-html (npm) 1.0.3 None
Copilot is powered by AI and may make mistakes. Always verify output.
Positive Feedback
Negative Feedback

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

Please select one or more of the options
logger.error(`POST /uploadExternalSource: Source's attributes are invalid`);
res.status(500);
if (sourceSchema !== undefined) {
res.send(`POST /uploadExternalSource: Source's attributes are invalid:\n${JSON.stringify(sourceSchema.errors)}`);

Check warning

Code scanning / CodeQL

Exception text reinterpreted as HTML Medium

JSON schema validation error
is reinterpreted as HTML without escaping meta-characters.

Copilot Autofix AI about 13 hours ago

To fix the problem, we need to ensure that any error messages sent back in the response are properly sanitized or escaped to prevent XSS attacks. The best way to fix this is to use a library like he (HTML entities) to escape any potentially dangerous characters in the error message before sending it back in the response.

  1. Install the he library to handle HTML entity encoding.
  2. Import the he library in the file.
  3. Use the he.encode function to escape the error message before sending it in the response.
Suggested changeset 2
src/packages/external-source/external-source.ts

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/src/packages/external-source/external-source.ts b/src/packages/external-source/external-source.ts
--- a/src/packages/external-source/external-source.ts
+++ b/src/packages/external-source/external-source.ts
@@ -9,2 +9,3 @@
 import { HasuraError } from '../../types/hasura.js';
+import he from 'he';
 
@@ -172,3 +173,3 @@
     if (sourceSchema !== undefined) {
-      res.send(`POST /uploadExternalSource: Source's attributes are invalid:\n${JSON.stringify(sourceSchema.errors)}`);
+      res.send(`POST /uploadExternalSource: Source's attributes are invalid:\n${he.encode(JSON.stringify(sourceSchema.errors))}`);
     } else {
EOF
@@ -9,2 +9,3 @@
import { HasuraError } from '../../types/hasura.js';
import he from 'he';

@@ -172,3 +173,3 @@
if (sourceSchema !== undefined) {
res.send(`POST /uploadExternalSource: Source's attributes are invalid:\n${JSON.stringify(sourceSchema.errors)}`);
res.send(`POST /uploadExternalSource: Source's attributes are invalid:\n${he.encode(JSON.stringify(sourceSchema.errors))}`);
} else {
package.json
Outside changed files

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/package.json b/package.json
--- a/package.json
+++ b/package.json
@@ -43,3 +43,4 @@
     "swagger-ui-express": "^4.6.3",
-    "winston": "^3.9.0"
+    "winston": "^3.9.0",
+    "he": "^1.2.0"
   },
EOF
@@ -43,3 +43,4 @@
"swagger-ui-express": "^4.6.3",
"winston": "^3.9.0"
"winston": "^3.9.0",
"he": "^1.2.0"
},
This fix introduces these dependencies
Package Version Security advisories
he (npm) 1.2.0 None
Copilot is powered by AI and may make mistakes. Always verify output.
Positive Feedback
Negative Feedback

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

Please select one or more of the options
} catch (e) {
logger.error(`POST /uploadExternalSource: ${(e as Error).message}`);
res.status(500);
res.send((e as Error).message);

Check failure

Code scanning / CodeQL

Reflected cross-site scripting High

Cross-site scripting vulnerability due to a
user-provided value
.

Copilot Autofix AI about 13 hours ago

To fix the reflected cross-site scripting vulnerability, we need to ensure that any user-provided data included in the error message is properly sanitized before being sent in the HTTP response. The best way to achieve this is by using a library like escape-html to escape any potentially dangerous characters in the error message.

  1. Import the escape-html library at the top of the file.
  2. Use the escape function from the escape-html library to sanitize the error message before sending it in the response.
Suggested changeset 2
src/packages/external-source/external-source.ts

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/src/packages/external-source/external-source.ts b/src/packages/external-source/external-source.ts
--- a/src/packages/external-source/external-source.ts
+++ b/src/packages/external-source/external-source.ts
@@ -1,2 +1,3 @@
 import type { Express, Request, Response } from 'express';
+import escape from 'escape-html';
 import type { DerivationGroupInsertInput, ExternalSourceInsertInput, ExternalSourceTypeInsertInput } from '../../types/external-source.js';
@@ -223,3 +224,3 @@
       res.status(500);
-      res.send((e as Error).message);
+      res.send(escape((e as Error).message));
       return;
EOF
@@ -1,2 +1,3 @@
import type { Express, Request, Response } from 'express';
import escape from 'escape-html';
import type { DerivationGroupInsertInput, ExternalSourceInsertInput, ExternalSourceTypeInsertInput } from '../../types/external-source.js';
@@ -223,3 +224,3 @@
res.status(500);
res.send((e as Error).message);
res.send(escape((e as Error).message));
return;
package.json
Outside changed files

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/package.json b/package.json
--- a/package.json
+++ b/package.json
@@ -43,3 +43,4 @@
     "swagger-ui-express": "^4.6.3",
-    "winston": "^3.9.0"
+    "winston": "^3.9.0",
+    "escape-html": "^1.0.3"
   },
EOF
@@ -43,3 +43,4 @@
"swagger-ui-express": "^4.6.3",
"winston": "^3.9.0"
"winston": "^3.9.0",
"escape-html": "^1.0.3"
},
This fix introduces these dependencies
Package Version Security advisories
escape-html (npm) 1.0.3 None
Copilot is powered by AI and may make mistakes. Always verify output.
Positive Feedback
Negative Feedback

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

Please select one or more of the options
} catch (e) {
logger.error(`POST /uploadExternalSource: ${(e as Error).message}`);
res.status(500);
res.send((e as Error).message);

Check warning

Code scanning / CodeQL

Exception text reinterpreted as HTML Medium

Exception text
is reinterpreted as HTML without escaping meta-characters.

Copilot Autofix AI about 13 hours ago

To fix the problem, we need to ensure that the error message is properly sanitized before being sent back in the response. This can be achieved by using a library like he (HTML entities) to escape any HTML meta-characters in the error message. This will prevent any malicious scripts from being executed in the user's browser.

  1. Install the he library to handle HTML entity encoding.
  2. Import the he library in the file.
  3. Use the he.encode function to sanitize the error message before sending it in the response.
Suggested changeset 2
src/packages/external-source/external-source.ts

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/src/packages/external-source/external-source.ts b/src/packages/external-source/external-source.ts
--- a/src/packages/external-source/external-source.ts
+++ b/src/packages/external-source/external-source.ts
@@ -9,2 +9,3 @@
 import { HasuraError } from '../../types/hasura.js';
+import he from 'he';
 
@@ -221,5 +222,6 @@
     } catch (e) {
-      logger.error(`POST /uploadExternalSource: ${(e as Error).message}`);
+      const sanitizedMessage = he.encode((e as Error).message);
+      logger.error(`POST /uploadExternalSource: ${sanitizedMessage}`);
       res.status(500);
-      res.send((e as Error).message);
+      res.send(sanitizedMessage);
       return;
EOF
@@ -9,2 +9,3 @@
import { HasuraError } from '../../types/hasura.js';
import he from 'he';

@@ -221,5 +222,6 @@
} catch (e) {
logger.error(`POST /uploadExternalSource: ${(e as Error).message}`);
const sanitizedMessage = he.encode((e as Error).message);
logger.error(`POST /uploadExternalSource: ${sanitizedMessage}`);
res.status(500);
res.send((e as Error).message);
res.send(sanitizedMessage);
return;
package.json
Outside changed files

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/package.json b/package.json
--- a/package.json
+++ b/package.json
@@ -43,3 +43,4 @@
     "swagger-ui-express": "^4.6.3",
-    "winston": "^3.9.0"
+    "winston": "^3.9.0",
+    "he": "^1.2.0"
   },
EOF
@@ -43,3 +43,4 @@
"swagger-ui-express": "^4.6.3",
"winston": "^3.9.0"
"winston": "^3.9.0",
"he": "^1.2.0"
},
This fix introduces these dependencies
Package Version Security advisories
he (npm) 1.2.0 None
Copilot is powered by AI and may make mistakes. Always verify output.
Positive Feedback
Negative Feedback

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

Please select one or more of the options
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant