-
Notifications
You must be signed in to change notification settings - Fork 299
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Custom headers ignored in HTTPCollectorBot #2150
Labels
bug
Indicates an unexpected problem or unintended behavior
component: bots
help wanted
Indicates that a maintainer wants help on an issue or pull request
Comments
See also #2137 |
sebix
added
bug
Indicates an unexpected problem or unintended behavior
component: bots
help wanted
Indicates that a maintainer wants help on an issue or pull request
labels
Feb 3, 2022
waldbauer-certat
added a commit
that referenced
this issue
Feb 4, 2022
**General** Removed 'requests' MissingDependencyError, because requests is a core lib from intelmq Removed HTTP variables from Bot class in favor of HttpMixin Removed trying to import requests in pipeline, its a core lib from intelmq Added additional configuration variables to HttpMixin ( from Bot class ) **Bots** GitHub API is now using HttpMixin MS Azure Collector is now using HttpMixin DO-Portal Expert is now using HttpMixin GeoHash using MissingDependencyError instead of ValueError (consistency) HttpContentExpert is now using HttpMixin HttpStatusExpert is now using HttpMixin NationalCERTContactCertATExpert is now using HttpMixin RDAPExpert is now using HttpMixin RIPEExpert is now using HttpMixin SplunkSavedSearchExpert is now using HttpMixin TuencyExpert is now using HttpMixin RestAPIOutput is now using HttpMixin **Bot tests** GitHub API Collector is now using requests_mock instead of MagicMock (consistency) RestAPI Output is now using correct headers Fixes #2150 Fixes #2137 Signed-off-by: Sebastian Waldbauer <[email protected]>
waldbauer-certat
added a commit
that referenced
this issue
Feb 4, 2022
**General** Removed 'requests' MissingDependencyError, because requests is a core lib from intelmq Removed HTTP variables from Bot class in favor of HttpMixin Removed trying to import requests in pipeline, its a core lib from intelmq Added additional configuration variables to HttpMixin ( from Bot class ) **Bots** GitHub API is now using HttpMixin MS Azure Collector is now using HttpMixin DO-Portal Expert is now using HttpMixin GeoHash using MissingDependencyError instead of ValueError (consistency) HttpContentExpert is now using HttpMixin HttpStatusExpert is now using HttpMixin NationalCERTContactCertATExpert is now using HttpMixin RDAPExpert is now using HttpMixin RIPEExpert is now using HttpMixin SplunkSavedSearchExpert is now using HttpMixin TuencyExpert is now using HttpMixin RestAPIOutput is now using HttpMixin **Bot tests** GitHub API Collector is now using requests_mock instead of MagicMock (consistency) RestAPI Output is now using correct headers Fixes #2150 Fixes #2137 Signed-off-by: Sebastian Waldbauer <[email protected]>
waldbauer-certat
added a commit
that referenced
this issue
Feb 4, 2022
**General** Removed 'requests' MissingDependencyError, because requests is a core lib from intelmq Removed HTTP variables from Bot class in favor of HttpMixin Removed trying to import requests in pipeline, its a core lib from intelmq Added additional configuration variables to HttpMixin ( from Bot class ) **Bots** GitHub API is now using HttpMixin MS Azure Collector is now using HttpMixin DO-Portal Expert is now using HttpMixin GeoHash using MissingDependencyError instead of ValueError (consistency) HttpContentExpert is now using HttpMixin HttpStatusExpert is now using HttpMixin NationalCERTContactCertATExpert is now using HttpMixin RDAPExpert is now using HttpMixin RIPEExpert is now using HttpMixin SplunkSavedSearchExpert is now using HttpMixin TuencyExpert is now using HttpMixin RestAPIOutput is now using HttpMixin **Bot tests** GitHub API Collector is now using requests_mock instead of MagicMock (consistency) RestAPI Output is now using correct headers Fixes #2150 Fixes #2137 Signed-off-by: Sebastian Waldbauer <[email protected]>
waldbauer-certat
added a commit
that referenced
this issue
Feb 4, 2022
**General** Removed 'requests' MissingDependencyError, because requests is a core lib from intelmq Removed HTTP variables from Bot class in favor of HttpMixin Removed trying to import requests in pipeline, its a core lib from intelmq Added additional configuration variables to HttpMixin ( from Bot class ) **Bots** GitHub API is now using HttpMixin MS Azure Collector is now using HttpMixin DO-Portal Expert is now using HttpMixin GeoHash using MissingDependencyError instead of ValueError (consistency) HttpContentExpert is now using HttpMixin HttpStatusExpert is now using HttpMixin NationalCERTContactCertATExpert is now using HttpMixin RDAPExpert is now using HttpMixin RIPEExpert is now using HttpMixin SplunkSavedSearchExpert is now using HttpMixin TuencyExpert is now using HttpMixin RestAPIOutput is now using HttpMixin **Bot tests** GitHub API Collector is now using requests_mock instead of MagicMock (consistency) RestAPI Output is now using correct headers Fixes #2150 Fixes #2137 Signed-off-by: Sebastian Waldbauer <[email protected]>
waldbauer-certat
added a commit
that referenced
this issue
Feb 4, 2022
**General** Removed 'requests' MissingDependencyError, because requests is a core lib from intelmq Removed HTTP variables from Bot class in favor of HttpMixin Removed trying to import requests in pipeline, its a core lib from intelmq Added additional configuration variables to HttpMixin ( from Bot class ) **Bots** GitHub API is now using HttpMixin MS Azure Collector is now using HttpMixin DO-Portal Expert is now using HttpMixin GeoHash using MissingDependencyError instead of ValueError (consistency) HttpContentExpert is now using HttpMixin HttpStatusExpert is now using HttpMixin NationalCERTContactCertATExpert is now using HttpMixin RDAPExpert is now using HttpMixin RIPEExpert is now using HttpMixin SplunkSavedSearchExpert is now using HttpMixin TuencyExpert is now using HttpMixin RestAPIOutput is now using HttpMixin **Bot tests** GitHub API Collector is now using requests_mock instead of MagicMock (consistency) RestAPI Output is now using correct headers Fixes #2150 Fixes #2137 Signed-off-by: Sebastian Waldbauer <[email protected]>
waldbauer-certat
added a commit
that referenced
this issue
Feb 4, 2022
**General** Removed 'requests' MissingDependencyError, because requests is a core lib from intelmq Removed HTTP variables from Bot class in favor of HttpMixin Removed trying to import requests in pipeline, its a core lib from intelmq Added additional configuration variables to HttpMixin ( from Bot class ) **Bots** GitHub API is now using HttpMixin MS Azure Collector is now using HttpMixin DO-Portal Expert is now using HttpMixin GeoHash using MissingDependencyError instead of ValueError (consistency) HttpContentExpert is now using HttpMixin HttpStatusExpert is now using HttpMixin NationalCERTContactCertATExpert is now using HttpMixin RDAPExpert is now using HttpMixin RIPEExpert is now using HttpMixin SplunkSavedSearchExpert is now using HttpMixin TuencyExpert is now using HttpMixin RestAPIOutput is now using HttpMixin **Bot tests** GitHub API Collector is now using requests_mock instead of MagicMock (consistency) RestAPI Output is now using correct headers Fixes #2150 Fixes #2137 Signed-off-by: Sebastian Waldbauer <[email protected]>
waldbauer-certat
added a commit
that referenced
this issue
Feb 7, 2022
**General** Removed 'requests' MissingDependencyError, because requests is a core lib from intelmq Removed HTTP variables from Bot class in favor of HttpMixin Removed trying to import requests in pipeline, its a core lib from intelmq Added additional configuration variables to HttpMixin ( from Bot class ) **Bots** GitHub API is now using HttpMixin MS Azure Collector is now using HttpMixin DO-Portal Expert is now using HttpMixin GeoHash using MissingDependencyError instead of ValueError (consistency) HttpContentExpert is now using HttpMixin HttpStatusExpert is now using HttpMixin NationalCERTContactCertATExpert is now using HttpMixin RDAPExpert is now using HttpMixin RIPEExpert is now using HttpMixin SplunkSavedSearchExpert is now using HttpMixin TuencyExpert is now using HttpMixin RestAPIOutput is now using HttpMixin **Bot tests** GitHub API Collector is now using requests_mock instead of MagicMock (consistency) RestAPI Output is now using correct headers Fixes #2150 Fixes #2137 Signed-off-by: Sebastian Waldbauer <[email protected]>
waldbauer-certat
added a commit
that referenced
this issue
Feb 7, 2022
**General** Removed 'requests' MissingDependencyError, because requests is a core lib from intelmq Removed HTTP variables from Bot class in favor of HttpMixin Removed trying to import requests in pipeline, its a core lib from intelmq Added additional configuration variables to HttpMixin ( from Bot class ) **Bots** GitHub API is now using HttpMixin MS Azure Collector is now using HttpMixin DO-Portal Expert is now using HttpMixin GeoHash using MissingDependencyError instead of ValueError (consistency) HttpContentExpert is now using HttpMixin HttpStatusExpert is now using HttpMixin NationalCERTContactCertATExpert is now using HttpMixin RDAPExpert is now using HttpMixin RIPEExpert is now using HttpMixin SplunkSavedSearchExpert is now using HttpMixin TuencyExpert is now using HttpMixin RestAPIOutput is now using HttpMixin **Bot tests** GitHub API Collector is now using requests_mock instead of MagicMock (consistency) RestAPI Output is now using correct headers Fixes #2150 Fixes #2137 Signed-off-by: Sebastian Waldbauer <[email protected]>
waldbauer-certat
added a commit
that referenced
this issue
Feb 7, 2022
**General** Removed 'requests' MissingDependencyError, because requests is a core lib from intelmq Removed HTTP variables from Bot class in favor of HttpMixin Removed trying to import requests in pipeline, its a core lib from intelmq Added additional configuration variables to HttpMixin ( from Bot class ) **Bots** GitHub API is now using HttpMixin MS Azure Collector is now using HttpMixin DO-Portal Expert is now using HttpMixin GeoHash using MissingDependencyError instead of ValueError (consistency) HttpContentExpert is now using HttpMixin HttpStatusExpert is now using HttpMixin NationalCERTContactCertATExpert is now using HttpMixin RDAPExpert is now using HttpMixin RIPEExpert is now using HttpMixin SplunkSavedSearchExpert is now using HttpMixin TuencyExpert is now using HttpMixin RestAPIOutput is now using HttpMixin **Bot tests** GitHub API Collector is now using requests_mock instead of MagicMock (consistency) RestAPI Output is now using correct headers Fixes #2150 Fixes #2137 Signed-off-by: Sebastian Waldbauer <[email protected]>
waldbauer-certat
added a commit
that referenced
this issue
Feb 7, 2022
**General** Removed 'requests' MissingDependencyError, because requests is a core lib from intelmq Removed HTTP variables from Bot class in favor of HttpMixin Removed trying to import requests in pipeline, its a core lib from intelmq Added additional configuration variables to HttpMixin ( from Bot class ) **Bots** GitHub API is now using HttpMixin MS Azure Collector is now using HttpMixin DO-Portal Expert is now using HttpMixin GeoHash using MissingDependencyError instead of ValueError (consistency) HttpContentExpert is now using HttpMixin HttpStatusExpert is now using HttpMixin NationalCERTContactCertATExpert is now using HttpMixin RDAPExpert is now using HttpMixin RIPEExpert is now using HttpMixin SplunkSavedSearchExpert is now using HttpMixin TuencyExpert is now using HttpMixin RestAPIOutput is now using HttpMixin **Bot tests** GitHub API Collector is now using requests_mock instead of MagicMock (consistency) RestAPI Output is now using correct headers Fixes #2150 Fixes #2137 Signed-off-by: Sebastian Waldbauer <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
bug
Indicates an unexpected problem or unintended behavior
component: bots
help wanted
Indicates that a maintainer wants help on an issue or pull request
The problem
When using the
HTTPCollectorBot
, it is possible to provide custom headers in the bot's configuration using thehttp_header
variable. However, when running the bot, these custom headers are ignored, i.e. not passed on in the request.Probable cause
The
HTTPCollectorBot
uses theHttpMixin
class. This class contains a line that sets the instance's variableself.http_header
to an empty dictionary. This overwrites possible custom headers set in the bot's configuration in the manager.Possible solutions
bot.py
there is a functionset_request_parameters
which looks like it is supposed to set the custom headers (among other things). It might be a better solution to move this function to theHttpMixin
and calling it at the start of thesetup
function in theHttpMixin
.The text was updated successfully, but these errors were encountered: