OSIO Gemini server workflow implementation

[samuzzal-choudhury]

Description

This user story scopes the development work required for OSIO Gemini. The staging or production deployed Gemini API server shall provide an endpoint /register to register a new repository to the system that in turn will initiate an analytics scan on that and email a report with the security vulnerability status of all direct and transitive dependencies.

Ecosystems To Be Supported: Maven, Python, Node and Go.

Task List

• Finalize the API workflow
• Develop Gemini API Server with /register endpoint. (Geetika, [13] OSIO Gemini server tasks #2493).
• Define required worker flows to process a scan in the backend. (Aagam/Sam)
• Develop a task to parse a registered Maven GitHub repository and identify all dependencies. (Aagam)
• osioAnalysisFlow: Generalize dependency_tree task to support different ecosystems. (Aagam)
• osioAnalysisFlow: Generalize unknown_deps_fetcher task to support different ecosystems. (Vasu,
  [13] OSIO Gemini workflow implemenation tasks #2494)
• osioAnalysisFlow: Implement dependency parser for node ecosystem. (Aagam)
• osioAnalysisFlow: Implement dependency parser for python ecosystem. (Aagam)
• osioAnalysisFlow: Ingest unknown dependencies. (Vasu, [13] OSIO Gemini workflow implemenation tasks #2494)
• osioAnalysisFlow: report generation task to collate CVE for all dependencies. (Aagam)
• osioAnalysisFlow: Implement dependency parser for go ecosystem (gopkg.lock support). (Aagam)
• osioAnalysisFlow: Implement dependency parser for go ecosystem (glide-lock support). (Aagam)
• osioAnalysisFlow: Support multiple manifest files of the same ecosystem (Aagam)
• osioAnalysisFlow: Create a GitHub issue with scan report post report generation (Sam).
• Create UX for report e-mail
• Gemini Server: Develop /report endpoint to return report generation task output (Vasu,[13] OSIO Gemini workflow implemenation tasks #2494)
• Deploy Gemini API Server with /register endpoint on production. (Geetika, [13] OSIO Gemini server tasks #2493)
• osioAnalysisFlow: report generation task to call license service to retrieve stack license. (Geetika, [13] OSIO Gemini server tasks #2493)
• Deploy worker-api with osioAnalysisFlow on production. (Aagam)
• osioAnalysisFlow: Identify and Configure an e-mail server.

Productize (Stretch for 147):

• Gemini Server: Develop /scan endpoint to initiate osioAnalysisFlow. (Aagam)
• Create a scheduled job to periodically run a scan on registered repositories.
• Gemini Server: Develop /scanner-error endpoint to report scanner level errors via e-mail.
• Gemini Server: Develop integration tests for Gemini server.
• osioAnalysisFlow: e-mail intended recipients via the configured mail server.
• Deploy Gemini service on the production cluster.
• Deploy worker API on the production cluster.

[geetikabatra]

Associated user story #2493

[maxandersen]

We have email notifications setup and api in platform this should use. Not sure if notifications api fits to this yet but we’ve gone through all the mail setup thus should be best to fix/use that.

examples of templates in notification service is at https://github.com/fabric8-services/fabric8-notification/tree/master/template. I realized we don't actually have docs on how/where to use this api so went and created fabric8-services/fabric8-notification#60

in any case, when you need more info raise it on devtools-saas@

[samuzzal-choudhury]

Sprint 147 user story for Epic #2774

[sivaavkd]

@abs51295 Can you close this and create a new story for 148 ? I believe the stretch items mentioned in this story will become the items for 148. cc @samuzzal-choudhury

[miteshvp]

Closing in favor of #3167