Sprint plan for Fabric8 Analytics: #146

[krishnapaparaju]

IMPROVE current Insights for WF Swam based boosters

• [3] Extend Insights for WildFly Swarm applications #2090

Ingest RHAT specific downstream dependencies present at OSIO boosters #2282

• Identify changes needed for supporting 3rd party package repositories [5]

Insights for Node ecosystem

• [13] Productionizing Deep learning models [Get model training to run on EMR) [13] Get the deep learning model to train on EMR through data collected from core #2467 (part of Productionize Deep learning Models for Node ecosystem #2434)

Scale the platform to support 500 concurrent users

• Extend the current test results for 500 concurrent users to include mixed traffic
  (New Stack analysis initiation, Retrieval of stack report, LSP calls) [8] Scale analytics platform to support 500 concurrent requests #2491 (Part of Scale OSIO analytics platform  #1144)
• Work with SD team to get necessary resources
• Promote changes to OSIO

Complete OSIO Gemini related activities related to Fabric8 Analytics integration #2326

• [13] OSIO Gemini workflow implemenation tasks #2494 [13]
• [13] OSIO Gemini server tasks #2493 [13]
• [8] Generalize GithubDependencyTreeTask to support multiple ecosystems #2496 [8]

Map CVEs for the Maven, Node ecosystems

• Continue the current manual mapping from NVDB to individual packages [1]
• Continue POC for automating CPEs to individual dependencies POC for automating mapping CPEs to individual application dependencies #2106
• Find a way how to get project name from NVD CVE data [8]
• Clean up Maven CVE false positives in graph database [5]

Close known SEV-2 bugs

• Stack report not showing up at dash board while a build is in progress #2335
• Closing stack report modal at space dash board screen disables entire screen #2328
• Current graph queries could be missing some package versions  #2313
• two vertical scrollbars when stack report modal is opened #2272
• No such bucket error encountered [3]
• psycopg2.IntegrityError
• Different 'label' values are stored in the Graph database for package versions data
• Typo in stack analysis v2 result JSON  #843
• Server Documentation and Readme corrections. #833
• Error while calling stack analyses from a github url #814

Improve service side for better handle on service deployment

• Resilient infra for services of Analytics platform
• Use jobs service as a gateway for communicating with data-importer
• Integrate data-importer with Sentry [3]
• Integrate f8a jobs service with Sentry [3]

Data ingestion, sanity related activities

• Scale the transfer of packages to graph database [5]
• All the known Maven, Node dependencies are present at Graph
• Ingest Node components which have keywords and GitHub topics
• Gradle support in f8a-worker

Quality

• [5] Implement Referential Integrity (RI) tests for the data stored in S3 database #1635 [5] Implement Referential Integrity (RI) tests for the data stored in S3 database
• [5] Implement Referential Integrity (RI) tests for the data stored in graph database #1636 [5] Implement Referential Integrity (RI) tests for the data stored in graph database
• [5] Improve test coverage in fabric8-analytics/fabric8-analytics-data-model repository #2256
• [2] RFE: Improve code quality in the following repository: fabric8-gemini-server #2454 [2] RFE: Improve code quality in the following repository: fabric8-gemini-server
• [1] RFE: Improve code quality in the following repository: fabric8-analytics-worker #1227 [1] RFE: Improve code quality in the following repository: fabric8-analytics-worker
• [3] Configure Pydocstyle in all relevant fabric8-analytics repositories #1089 [3] Configure Pydocstyle in all relevant fabric8-analytics repositories
• [4] New integration tests: checks for component and analysis results stored in the S3 #1077 [4]
• [3] Provide unit tests for the sources from repository fabric8-analytics-tagger #1744 [3] Provide unit tests for the sources from repository fabric8-analytics-tagger
• [1] Improve QA Dashboard to support list of files that are ignored by pylinter or pydocstyle checker #2497 [1] Improve QA Dashboard to support list of files that are ignored by pylinter or pydocstyle checker
• [1] Implement a tool to display code coverage weekly status #2750 [1] Implement a tool to display code coverage weekly status