-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
20c11aa
commit d3445d0
Showing
8 changed files
with
304 additions
and
43 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -22,3 +22,4 @@ coverage | |
# Files # | ||
################### | ||
*.log | ||
__pycache__ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -8,17 +8,39 @@ | |
[![license](https://img.shields.io/github/license/forwardemail/python-arf.svg)](LICENSE) | ||
[![npm downloads](https://img.shields.io/npm/dt/python-arf.svg)](https://npm.im/python-arf) | ||
|
||
> Node.js wrapper around the Python package arf, which is a processor for Abu se Reporting Format (ARF) messages. | ||
> Node.js wrapper around the Python package arf, which is a processor for Abuse Reporting Format (ARF) messages. | ||
|
||
## Table of Contents | ||
|
||
* [Requirements](#requirements) | ||
* [Install](#install) | ||
* [Usage](#usage) | ||
* [Contributors](#contributors) | ||
* [License](#license) | ||
|
||
|
||
## Requirements | ||
|
||
1. Ensure that you have a Python version of >= 3.5 installed (we do not support older versions of Python anymore across the entire organization): | ||
|
||
```sh | ||
python3 --version | ||
``` | ||
|
||
2. Install the package [arf](https://github.com/danielsen/arf) ([not yet available in pip](https://github.com/danielsen/arf/issues/1)): | ||
|
||
```sh | ||
# | ||
# NOTE: we cannot use `git clone https://github.com/danielsen/arf.git` | ||
# until <https://github.com/danielsen/arf/pull/3> is merged | ||
git clone https://github.com/niftylettuce/arf.git | ||
cd arf | ||
git checkout patch-1 | ||
python3 setup.py install | ||
``` | ||
|
||
|
||
## Install | ||
|
||
[npm][]: | ||
|
@@ -36,15 +58,87 @@ yarn add python-arf | |
|
||
## Usage | ||
|
||
Note that the `source` is an ARF message and can be a file path (String), String, or Buffer. | ||
|
||
```js | ||
const PythonArf = require('python-arf'); | ||
const arf = require('python-arf'); | ||
// then/catch usage | ||
arf(source) | ||
.then(console.log) | ||
.catch(console.error); | ||
// async/await usage | ||
(async () => { | ||
try { | ||
const result = await arf(source); | ||
console.log(result); | ||
} catch (err) { | ||
console.error(err); | ||
} | ||
})(); | ||
``` | ||
|
||
const pythonArf = new PythonArf(); | ||
Note that `result` is an Object that looks like this: | ||
|
||
console.log(pythonArf.renderName()); | ||
// script | ||
```js | ||
{ | ||
MessageHeaders: { | ||
Received: [ | ||
'from example.com (example.com [10.0.1.11]) by example.net with ESMTP id O63d4137594e46; Thu, 08 Mar 2005 16:00:00 -0400', | ||
'from mailserver.example.net (mailserver.example.net [192.0.2.1]) by example.com with ESMTP id M63d4137594e46; Thu, 08 Mar 2005 14:00:00 -0400' | ||
], | ||
From: '<[email protected]>', | ||
Date: 'Thu, 8 Mar 2005 17:40:36 EDT', | ||
Subject: 'FW: Earn money', | ||
To: '<[email protected]>', | ||
MimeVersion: '1.0', | ||
ContentType: 'multipart/report; report-type=feedback-report; boundary="part1_13d.2e68ed54_boundary"' | ||
}, | ||
OriginalMessageHeaders: { | ||
From: '<[email protected]>', | ||
Received: 'from mailserver.example.net (mailserver.example.net [192.0.2.1]) by example.com with ESMTP id M63d4137594e46; Thu, 08 Mar 2005 14:00:00 -0400', | ||
To: '<Undisclosed Recipients>', | ||
Subject: 'Earn money', | ||
MimeVersion: '1.0', | ||
ContentType: 'text/plain', | ||
MessageId: '[email protected]', | ||
Date: 'Thu, 02 Sep 2004 12:31:03 -0500' | ||
}, | ||
FeedbackReport: { | ||
FeedbackType: 'abuse', | ||
UserAgent: 'SomeGenerator/1.0', | ||
Version: '1', | ||
OriginalMailFrom: '<[email protected]>', | ||
OriginalRcptTo: '<[email protected]>', | ||
ArrivalDate: 'Thu, 8 Mar 2005 14:00:00 EDT', | ||
ReportingMta: 'dns; mail.example.com', | ||
SourceIp: '192.0.2.1', | ||
AuthenticationResults: 'mail.example.com; spf=fail [email protected]', | ||
ReportedDomain: 'example.net', | ||
ReportedUri: [ 'http://example.net/earn_money.html', 'mailto:[email protected]' ], | ||
RemovalRecipient: '[email protected]' | ||
}, | ||
OriginalMessagePayload: 'From: <[email protected]>\n' + | ||
'Received: from mailserver.example.net (mailserver.example.net\n' + | ||
'\t[192.0.2.1]) by example.com with ESMTP id M63d4137594e46;\n' + | ||
'\tThu, 08 Mar 2005 14:00:00 -0400\n' + | ||
'To: <Undisclosed Recipients>\n' + | ||
'Subject: Earn money\n' + | ||
'MIME-Version: 1.0\n' + | ||
'Content-type: text/plain\n' + | ||
'Message-ID: [email protected]\n' + | ||
'Date: Thu, 02 Sep 2004 12:31:03 -0500\n' + | ||
'\n' + | ||
'Spam Spam Spam\n' + | ||
'Spam Spam Spam\n' + | ||
'Spam Spam Spam\n' + | ||
'Spam Spam Spam' | ||
} | ||
``` | ||
|
||
Therefore if you wanted to access the original message payload in the ARF report (which is what we needed to do in [Forward Email](https://forwardemail.net), then you'd retrieve it via `result.OriginalMessagePayload`. | ||
## Contributors | ||
|
@@ -58,7 +152,7 @@ console.log(pythonArf.renderName()); | |
[MIT](LICENSE) © [Nick Baugh](http://niftylettuce.com/) | ||
## | ||
## | ||
[npm]: https://www.npmjs.com/ | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
#!/usr/bin/env python | ||
|
||
import json | ||
import sys | ||
import arf | ||
|
||
# message = arf.ARFMessage(sys.argv[1]).serialize_report_to_json() | ||
|
||
load = arf.load_arf(sys.argv[1]) | ||
message = load.serialize_report_to_json() | ||
message = json.loads(message) | ||
message['OriginalMessagePayload'] = load.get_original_message_payload() | ||
print(json.dumps(message)) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,14 +1,47 @@ | ||
class Script { | ||
constructor(config) { | ||
config = { ...config }; | ||
this._name = config.name || 'script'; | ||
const fs = require('fs'); | ||
const os = require('os'); | ||
const path = require('path'); | ||
|
||
this.renderName = this.renderName.bind(this); | ||
} | ||
const isBuffer = require('is-buffer'); | ||
const isValidPath = require('is-valid-path'); | ||
const revHash = require('rev-hash'); | ||
const semver = require('semver'); | ||
const { which, exec } = require('shelljs'); | ||
|
||
const filePath = path.join(__dirname, '_arf.py'); | ||
|
||
// ensure python installed | ||
if (!which('python3')) throw new Error(`Python v3.5+ is required`); | ||
|
||
const silent = process.env.NODE_ENV !== 'test'; | ||
|
||
renderName() { | ||
return this._name; | ||
// ensure python v3.5+ | ||
let version = exec('python3 --version', { silent }); | ||
version = semver.coerce( | ||
(version.stdout || version.stderr).split(' ')[1].trim() | ||
); | ||
|
||
if (!semver.satisfies(version, '>= 3.5')) | ||
throw new Error( | ||
`Python v3.5+ is required, you currently have v${version} installed` | ||
); | ||
|
||
async function arf(str) { | ||
if (!isBuffer(str) && typeof str !== 'string') | ||
throw new Error('str must be a buffer or string/path'); | ||
|
||
if (isBuffer(str) || (typeof str === 'string' && !isValidPath(str))) { | ||
const tmpPath = path.join(os.tmpdir(), revHash(str)); | ||
await fs.promises.writeFile(tmpPath, str); | ||
str = tmpPath; | ||
} | ||
|
||
return new Promise((resolve, reject) => { | ||
exec(`python3 ${filePath} ${str}`, { silent }, (code, stdout, stderr) => { | ||
if (code !== 0) return reject(new Error(stderr)); | ||
resolve(JSON.parse(stdout.trim())); | ||
}); | ||
}); | ||
} | ||
|
||
module.exports = Script; | ||
module.exports = arf; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,16 +1,25 @@ | ||
{ | ||
"name": "python-arf", | ||
"description": "Node.js wrapper around the Python package arf, which is a processor for Abu se Reporting Format (ARF) messages.", | ||
"description": "Node.js wrapper around the Python package arf, which is a processor for Abuse Reporting Format (ARF) messages.", | ||
"version": "0.0.0", | ||
"author": "Nick Baugh <[email protected]> (http://niftylettuce.com/)", | ||
"ava": { | ||
"verbose": true | ||
}, | ||
"bugs": { | ||
"url": "https://github.com/forwardemail/python-arf/issues", | ||
"email": "[email protected]" | ||
}, | ||
"contributors": [ | ||
"Nick Baugh <[email protected]> (http://niftylettuce.com/)" | ||
], | ||
"dependencies": {}, | ||
"dependencies": { | ||
"is-buffer": "^2.0.4", | ||
"is-valid-path": "^0.1.1", | ||
"rev-hash": "^3.0.0", | ||
"semver": "^7.3.2", | ||
"shelljs": "^0.8.4" | ||
}, | ||
"devDependencies": { | ||
"@commitlint/cli": "latest", | ||
"@commitlint/config-conventional": "latest", | ||
|
@@ -34,12 +43,21 @@ | |
"keywords": [ | ||
"abuse", | ||
"arf", | ||
"f", | ||
"email", | ||
"emails", | ||
"feedback", | ||
"format", | ||
"formats", | ||
"message", | ||
"messages", | ||
"parse", | ||
"parser", | ||
"parsing", | ||
"pip", | ||
"python", | ||
"report", | ||
"reporting" | ||
"reporting", | ||
"wrapper" | ||
], | ||
"license": "MIT", | ||
"main": "index.js", | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,60 @@ | ||
Received: from example.com (example.com [10.0.1.11]) | ||
by example.net with ESMTP id O63d4137594e46; | ||
Thu, 08 Mar 2005 16:00:00 -0400 | ||
Received: from mailserver.example.net (mailserver.example.net | ||
[192.0.2.1]) by example.com with ESMTP id M63d4137594e46; | ||
Thu, 08 Mar 2005 14:00:00 -0400 | ||
From: <[email protected]> | ||
Date: Thu, 8 Mar 2005 17:40:36 EDT | ||
Subject: FW: Earn money | ||
To: <[email protected]> | ||
MIME-Version: 1.0 | ||
Content-Type: multipart/report; report-type=feedback-report; | ||
boundary="part1_13d.2e68ed54_boundary" | ||
|
||
--part1_13d.2e68ed54_boundary | ||
Content-Type: text/plain; charset="US-ASCII" | ||
Content-Transfer-Encoding: 7bit | ||
This is an email abuse report for an email message received from IP | ||
192.0.2.1 on Thu, 8 Mar 2005 14:00:00 EDT. For more information | ||
about this format please see http://www.mipassoc.org/arf/. | ||
--part1_13d.2e68ed54_boundary | ||
Content-Type: message/feedback-report | ||
Feedback-Type: abuse | ||
User-Agent: SomeGenerator/1.0 | ||
Version: 1 | ||
Original-Mail-From: <[email protected]> | ||
Original-Rcpt-To: <[email protected]> | ||
Arrival-Date: Thu, 8 Mar 2005 14:00:00 EDT | ||
Reporting-MTA: dns; mail.example.com | ||
Source-IP: 192.0.2.1 | ||
Authentication-Results: mail.example.com; | ||
spf=fail [email protected] | ||
Reported-Domain: example.net | ||
Reported-Uri: http://example.net/earn_money.html | ||
Reported-Uri: mailto:[email protected] | ||
Removal-Recipient: [email protected] | ||
--part1_13d.2e68ed54_boundary | ||
Content-Type: message/rfc822 | ||
Content-Disposition: inline | ||
From: <[email protected]> | ||
Received: from mailserver.example.net (mailserver.example.net | ||
[192.0.2.1]) by example.com with ESMTP id M63d4137594e46; | ||
Thu, 08 Mar 2005 14:00:00 -0400 | ||
To: <Undisclosed Recipients> | ||
Subject: Earn money | ||
MIME-Version: 1.0 | ||
Content-type: text/plain | ||
Message-ID: [email protected] | ||
Date: Thu, 02 Sep 2004 12:31:03 -0500 | ||
Spam Spam Spam | ||
Spam Spam Spam | ||
Spam Spam Spam | ||
Spam Spam Spam | ||
--part1_13d.2e68ed54_boundary-- |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,27 +1,17 @@ | ||
const test = require('ava'); | ||
|
||
const Script = require('..'); | ||
|
||
test.beforeEach(t => { | ||
const script = new Script({}); | ||
Object.assign(t.context, { script }); | ||
}); | ||
const path = require('path'); | ||
|
||
test('returns itself', t => { | ||
t.true(t.context.script instanceof Script); | ||
}); | ||
const test = require('ava'); | ||
|
||
test('sets a config object', t => { | ||
const script = new Script(false); | ||
t.true(script instanceof Script); | ||
}); | ||
const arf = require('..'); | ||
|
||
test('renders name', t => { | ||
const { script } = t.context; | ||
t.is(script.renderName(), 'script'); | ||
test('throws error', async t => { | ||
await t.throwsAsync(arf()); | ||
t.pass(); | ||
}); | ||
|
||
test('sets a default name', t => { | ||
const { script } = t.context; | ||
t.is(script._name, 'script'); | ||
test('returns json', async t => { | ||
// <https://raw.githubusercontent.com/danielsen/arf/master/test/resources/sample_arf_message.txt> | ||
const result = await arf(path.join(__dirname, 'test.eml')); | ||
t.log(result); | ||
t.true(typeof result === 'object'); | ||
}); |
Oops, something went wrong.