forked from fabric8-analytics/fabric8-analytics-common
-
Notifications
You must be signed in to change notification settings - Fork 0
/
stack_analyses_v2.feature
190 lines (167 loc) · 8.59 KB
/
stack_analyses_v2.feature
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
Feature: Stack analysis v2 API
@sav2
Scenario Outline: Check that the stack analysis REST API endpoint for <method> HTTP method without user key
Given System is running
When I access the /api/v2/stack-analyses endpoint using the HTTP <method> method without user key
Then I should not get 200 status code
Examples: HTTP methods
| method |
| GET |
| HEAD |
| PUT |
| DELETE |
@sav2
Scenario Outline: Check that the stack analysis REST API endpoint for <method> HTTP method with user key
Given System is running
Given System is running
Given Three scale preview service is running
When I acquire the use_key for 3scale
Then I should get the proper user_key
When I access the /api/v2/stack-analyses endpoint using the HTTP <method> method with user key
Then I should not get 200 status code
Examples: HTTP methods
| method |
| GET |
| HEAD |
| PUT |
| DELETE |
@sav2
Scenario Outline: Check that the REST API enpoint <endpoint> for the stack analyses
Given System is running
When I access <endpoint>
Then I should get 401 status code
Examples: Endpoints and token
| endpoint |
| /api/v2/stack-analyses |
| /api/v2/stack-analyses/external_id |
@sav2
Scenario: Check that the API entry point without user key
Given System is running
When I send pypi package request with manifest valid_pylist.json to stack analysis v2 without valid user key
Then I should get 403 status code
@sav2
Scenario: Check that the API entry point with invalid user key
Given System is running
Given Three scale preview service is running
When I acquire the use_key for 3scale
Then I should get the proper user_key
When I send pypi package request with manifest valid_pylist.json to stack analysis v2 with invalid user key
Then I should get 403 status code
@sav2
Scenario: Check that the API entry point with user key
Given System is running
Given Three scale preview service is running
When I acquire the use_key for 3scale
Then I should get the proper user_key
When I wait 10 seconds
When I send pypi package request with manifest valid_pylist.json to stack analysis v2 with valid user key
Then I should get 200 status code
@sav2
Scenario Outline: Check the stack analysis v2 for <ecosystem> package and <manifest> manifest with valid user key
Given System is running
Given Three scale preview service is running
When I acquire the use_key for 3scale
Then I should get the proper user_key
When I wait 10 seconds
When I send <ecosystem> package request with manifest <manifest> to stack analysis v2 with valid user key
Then I should get 200 status code
And I should receive JSON response containing the status key
And I should receive JSON response containing the id key
And I should receive JSON response containing the submitted_at key
And I should receive JSON response with the status key set to success
And I should receive JSON response with the correct id
And I should receive JSON response with the correct timestamp in attribute submitted_at
Examples: Stack analyses POST params
| ecosystem | manifest |
| pypi | valid_pylist.json |
| npm | valid_npmlist.json |
| maven | valid_dependencies.txt |
@sav2
Scenario Outline: Check the stack analysis v2 with invalid request parameters
Given System is running
Given Three scale preview service is running
When I acquire the use_key for 3scale
Then I should get the proper user_key
When I wait 10 seconds
When I send <ecosystem> package request with manifest <manifest> to stack analysis v2 with valid user key
Then I should get 400 status code
Examples: Stack analyses POST params
| ecosystem | manifest |
| None | valid_pylist.json |
| npm | None |
| None | None |
| golang | None |
@sav2
Scenario Outline: Check the stack analysis v2 for invalid manifest data
Given System is running
Given Three scale preview service is running
When I acquire the use_key for 3scale
Then I should get the proper user_key
When I send <ecosystem> package request with manifest <manifest> to stack analysis v2 with valid user key
Then I should get 400 status code
Examples: Stack analyses POST params
| ecosystem | manifest |
| pypi | valid_npmlist.json |
| npm | valid_dependencies.txt |
@sav2
Scenario Outline: Check the stack analysis v2 request and response for proper data
Given System is running
Given Three scale preview service is running
When I acquire the use_key for 3scale
Then I should get the proper user_key
When I wait 10 seconds
When I send <ecosystem> package request with manifest <manifest> to stack analysis v2 with valid user key
Then I should get 200 status code
When I wait for stack analysis v2 to finish with user key
Then I should find the external request id equals to id returned by stack analysis v2 post request
And I should get stack analyses v2 response with all attributes
Examples: Stack analyses POST params
| ecosystem | manifest |
| pypi | valid_pylist.json |
| npm | valid_npmlist.json |
| maven | valid_dependencies.txt |
@sav2
Scenario Outline: Check the stack analysis v2 with known package data
Given System is running
Given Three scale preview service is running
When I acquire the use_key for 3scale
Then I should get the proper user_key
When I wait 10 seconds
When I send <ecosystem> package request with manifest <manifest> to stack analysis v2 with valid user key
Then I should get 200 status code
When I wait for stack analysis v2 to finish with user key
Then I should find the value <field> under the path <field_path> in the JSON response
And I should find the value <version> under the path <version_path> in the JSON response
Examples: Stack analyses POST params
| ecosystem | manifest | field | field_path | version | version_path |
| pypi | requests_2_20_0.json | requests | analyzed_dependencies/0/name | 2.20.0 | analyzed_dependencies/0/version |
| npm | npm_svg_2_0_2.json | svg.filter.js | analyzed_dependencies/0/name | 2.0.2 | analyzed_dependencies/0/version |
| maven | vertx_dependencies.txt | io.vertx:vertx-core | analyzed_dependencies/0/name | 3.4.1 | analyzed_dependencies/0/version |
@sav2 @skip
Scenario: Check the outlier record for a known package
Given System is running
Given Three scale preview service is running
When I acquire the use_key for 3scale
Then I should get the proper user_key
When I send maven package request with manifest springboot_dependencies.txt to stack analysis v2 with valid user key
Then I should get 200 status code
When I wait for stack analysis v2 to finish with user key
Then I should get stack analyses v2 response with all attributes
And I should find the proper outlier record for the org.springframework:spring-websocket component for stack analyses v2
@sav2
Scenario Outline: Check stack ananlyses v2 for vulnerabilities count for <ecosystem> with <manifest> manifest
Given System is running
Given Three scale preview service is running
When I acquire the use_key for 3scale
Then I should get the proper user_key
When I send <ecosystem> package request with manifest <manifest> to stack analysis v2 with valid user key
Then I should get 200 status code
When I wait for stack analysis v2 to finish with user key
Then I should get <public_count> public vulnerabilities for <component>
And I should get <private_count> private vulnerabilities for <component>
And I should get <transitive_count> transitive vulnerabilities for <component>
Examples: Stack analyses params
| ecosystem | manifest | component | public_count | private_count | transitive_count |
| pypi | fastlog_urllib_requests.json | requests | 1 | 0 | 0 |
| npm | npm_50_direct_799_tr.json | npm | 3 | 0 | 0 |
| maven | vertx_dependencies.txt | io.vertx:vertx-core | 2 | 0 | 0 |