Integrate with the automated API testing framework

.gitignore

@@ -7,3 +7,6 @@ scripts/generate
 .env
 coverage
 .vscode
+
+# Newman tests
+newman

README.md

@@ -14,6 +14,8 @@ Configuration for the application is at config/default.js and config/production.
 
 - LOG_LEVEL: the log level
 - PORT: the server port
+- API_BASE_URL: the api base url, default to 'http://127.0.0.1:3001/api/1.0'
+- AUTOMATED_TESTING_NAME_PREFIX: the prefix for the records from db, default to 'POSTMANE2E-',
 - CASCADE_PAUSE_MS: how many milliseconds to pause between deleting records during cascade delete (default to 1000)
 - AUTH_SECRET: TC Authentication secret
 - VALID_ISSUERS: valid issuers for TC authentication
@@ -58,7 +60,38 @@ Configuration for the application is at config/default.js and config/production.
 
 For `ES.DOCUMENTS` configuration, you will find multiple other configurations below it. Each has default values that you can override using the environment variables
 
-
+Configuration for testing is at `config/test.js`, only add such new configurations different from `config/default.js`
+- WAIT_TIME: wait time used in test, default is 0 or 0 second (5000 stands for 5 seconds)
+- AUTH_V2_URL: The auth v2 url
+- AUTH_V2_CLIENT_ID: The auth v2 client id
+- AUTH_V3_URL: The auth v3 url
+- ADMIN_CREDENTIALS_USERNAME: The user's username with admin role
+- ADMIN_CREDENTIALS_PASSWORD: The user's password with admin role
+- USER_CREDENTIALS_USERNAME: The user's username with user role
+- USER_CREDENTIALS_PASSWORD: The user's password with user role
+- MANAGER_CREDENTIALS_USERNAME: The user's username with manager role
+- MANAGER_CREDENTIALS_PASSWORD: The user's password with manager role
+- COPILOT_CREDENTIALS_USERNAME: The user's username with copilot role
+- COPILOT_CREDENTIALS_PASSWORD: The user's password with copilot role
+- USER_ID_BY_ADMIN: the user id which is created by admin
+- USER_ID_BY_TESTER: the user id which is created by a normal user
+- PROVIDER_ID_BY_ADMIN: the service provider id which is created by admin
+- PROVIDER_ID_BY_TESTER: the service provider id which is created by a normal user
+- SKILL_ID_BY_ADMIN: the skill id which is created by admin
+- SKILL_ID_BY_TESTER: the skill id which is created by a normal user
+- ROLE_ID_BY_ADMIN: the role id which is created by organization
+- ROLE_ID_BY_TESTER: the role id which is created by a normal user
+- ORGANIZATION_ID_BY_ADMIN: the organization id which is created by admin
+- ORGANIZATION_ID_BY_TESTER: the organization id which is created by a normal user
+- ACHIEVEMENTS_PROVIDER_ID_BY_ADMIN: the achievement provider id which is created by admin
+- ACHIEVEMENTS_PROVIDER_ID_BY_TESTER: the achievement provider id which is created by a normal user
+- ATTRIBUTE_GROUP_ID_BY_ADMIN: the attribute group id which is created by admin
+- ATTRIBUTE_GROUP_ID_BY_TESTER: the attribute group id which is created by a normal user
+- ATTRIBUTE_ID_BY_ADMIN: the attribute id which is created by admin
+- ATTRIBUTE_ID_BY_TESTER: the attribute id which is created by a normal user
+- ACHIEVEMENT_ID_BY_ADMIN: the achievement id which is created by admin
+- ACHIEVEMENT_ID_BY_TESTER: the achievement id which is created by a normal user
+- AUTOMATED_TESTING_REPORTERS_FORMAT: The format of output for Newman Automation Testing Report.
 ## Local deployment
 
 Setup your Elasticsearch instance and ensure that it is up and running.
@@ -105,3 +138,72 @@ Make sure all config values are right, and you can run on local successfully, th
 5. When you are running the application for the first time, It will take some time initially to download the image and install the dependencies
 
 You can also head into `docker-pgsql-es` folder and run `docker-compose up -d` to have docker instances of pgsql and elasticsearch to use with the api
+
+## Running tests
+
+### Configuration
+Test configuration is at `config/test.js`. You don't need to change them.
+
+The following test parameters can be set in config file or in env variables:
+
+- WAIT_TIME: wait time
+- AUTH_V2_URL: The auth v2 url
+- AUTH_V2_CLIENT_ID: The auth v2 client id
+- AUTH_V3_URL: The auth v3 url
+- ADMIN_CREDENTIALS_USERNAME: The user's username with admin role
+- ADMIN_CREDENTIALS_PASSWORD: The user's password with admin role
+- USER_CREDENTIALS_USERNAME: The user's username with user role
+- USER_CREDENTIALS_PASSWORD: The user's password with user role
+- COPILOT_CREDENTIALS_USERNAME: The user's username with copilot role
+- COPILOT_CREDENTIALS_PASSWORD: The user's password with copilot role
+- MANAGER_CREDENTIALS_USERNAME: The user's username with manager role
+- MANAGER_CREDENTIALS_PASSWORD: The user's password with manager role
+
+### Prepare
+
+- Start Postgres.
+- Create DynamoDB tables.
+- Start Local ElasticSearch.
+- Create ElasticSearch index.
+- Various config parameters should be properly set.
+
+### Running E2E tests with Postman
+
+#### `Start` the app server before running e2e tests. You may need to set the env variables by calling `source env.sh` before calling `NODE_ENV=test npm start`.
+
+- Make sure the db and es are properly started
+```bash
+  $ cd u-bahn-api
+
+    # NOTE:
+    # if tables and data already exist, please run first
+
+    # $ npm run delete-data
+
+    # to drop data and tables
+
+    # Then re-initialize the es server and the database. (Note, You need to drop the `SequelizeMeta` table manually.)
+
+  $ npm run migrations up            # to create table
+  $ npm run insert-postman-data      # to insert the testing data for postman
+  $ npm run migrate-db-to-es         # to index the db data to the ES
+```
+
+To run postman e2e tests.
+
+```bash
+npm run test:newman
+```
+
+To clear the testing data from postman e2e tests.
+
+```bash
+npm run test:newman:clear
+```
+
+## Running tests in CI
+- TBD
+
+## Verification
+
+Refer to the verification document `Verification.md`.

Verification.md

@@ -0,0 +1,93 @@
+# # U-Bahn API
+
+You can start the local pgsql and es by running `docker-compose up` from `docker-pgsql-es` folder. You should create a `.env` file and correctly set it.
+
+## E2E testing with Postman
+
+You should be able to find the tests result from the command window of running `npm run test:newman` for each test case.
+
+Below is a sample output result of finding users by a normal user.
+
+```
+Ubahn-api
+
+Iteration 1/9
+
+❏ users / search users
+↳ search users with various parameters by user
+  GET http://127.0.0.1:3001/api/1.0/users?page=1&perPage=10&enrich=true [200 OK, 2.28KB, 31ms]
+  ✓  Status code is 200
+
+Iteration 2/9
+
+↳ search users with various parameters by user
+  GET http://127.0.0.1:3001/api/1.0/users?page=1&perPage=1 [200 OK, 822B, 11ms]
+  ✓  Status code is 200
+
+Iteration 3/9
+
+↳ search users with various parameters by user
+  GET http://127.0.0.1:3001/api/1.0/users?page=1&perPage=1&handle=POSTMANE2E-lazybaer [200 OK, 424B, 10ms]
+  ✓  Status code is 200
+
+Iteration 4/9
+
+↳ search users with various parameters by user
+  GET http://127.0.0.1:3001/api/1.0/users?roleId=977d66bb-112f-49b1-8b20-2ef7a17f4943 [200 OK, 425B, 9ms]
+  ✓  Status code is 200
+
+Iteration 5/9
+
+↳ search users with various parameters by user
+  GET http://127.0.0.1:3001/api/1.0/users?enrich=true&externalProfile.externalId=8547899 [200 OK, 425B, 10ms]
+  ✓  Status code is 200
+
+Iteration 6/9
+
+↳ search users with various parameters by user
+  GET http://127.0.0.1:3001/api/1.0/users?enrich=true&externalProfile.organizationId=36ed815b-3da1-49f1-a043-aaed0a4e81ad [200 OK, 2.39KB, 11ms]
+  ✓  Status code is 200
+
+Iteration 7/9
+
+↳ search users with various parameters by user
+  GET http://127.0.0.1:3001/api/1.0/users?page=1&perPage=10&enrich=true&handle=POSTMANE2E-tc-Admin&roleId=977d66bb-112f-49b1-8b20-2ef7a17f4943&externalProfile.externalId=8547899&externalProfile.organizationId=36ed815b-3da1-49f1-a043-aaed0a4e81ad [200 OK, 425B, 14ms]
+  ✓  Status code is 200
+
+Iteration 8/9
+
+↳ search users with various parameters by user
+  GET http://127.0.0.1:3001/api/1.0/users?handle=POSTMANE2E-Sylvan_Gorczany [200 OK, 425B, 12ms]
+  ✓  Status code is 200
+
+Iteration 9/9
+
+↳ search users with various parameters by user
+  GET http://127.0.0.1:3001/api/1.0/users?handle=POSTMANE2E-Dana_Paucek6 [200 OK, 425B, 11ms]
+  ✓  Status code is 200
+
+┌─────────────────────────┬──────────────────┬─────────────────┐
+│                         │         executed │          failed │
+├─────────────────────────┼──────────────────┼─────────────────┤
+│              iterations │                9 │               0 │
+├─────────────────────────┼──────────────────┼─────────────────┤
+│                requests │                9 │               0 │
+├─────────────────────────┼──────────────────┼─────────────────┤
+│            test-scripts │                9 │               0 │
+├─────────────────────────┼──────────────────┼─────────────────┤
+│      prerequest-scripts │                9 │               0 │
+├─────────────────────────┼──────────────────┼─────────────────┤
+│              assertions │                9 │               0 │
+├─────────────────────────┴──────────────────┴─────────────────┤
+│ total run duration: 479ms                                    │
+├──────────────────────────────────────────────────────────────┤
+│ total data received: 3.66KB (approx)                         │
+├──────────────────────────────────────────────────────────────┤
+│ average response time: 13ms [min: 9ms, max: 31ms, s.d.: 6ms] │
+└──────────────────────────────────────────────────────────────┘
+```
+
+Then you can run `npm run test:newman:clear` to delete all testing data by above postman tests.  
+
+If 'socket hang up' appears while running the `npm run test:newman`. You can increase the `WAIT_TIME` from the `default/test.js`.
+Then run `npm run test:newman:clear`, `npm run insert-postman-data` and `npm run migrate-db-to-es` before calling `npm run test:newman` again.

config/default.js

@@ -5,12 +5,14 @@
 module.exports = {
   LOG_LEVEL: process.env.LOG_LEVEL || 'debug',
   PORT: process.env.PORT || 3001,
+  API_BASE_URL: process.env.API_BASE_URL || 'http://127.0.0.1:3001/api/1.0',
+  AUTOMATED_TESTING_NAME_PREFIX: process.env.AUTOMATED_TESTING_NAME_PREFIX || 'POSTMANE2E-',
 
   CASCADE_PAUSE_MS: process.env.CASCADE_PAUSE_MS || 1000,
 
   AUTH_SECRET: process.env.AUTH_SECRET || 'CLIENT_SECRET',
   VALID_ISSUERS: process.env.VALID_ISSUERS ? process.env.VALID_ISSUERS.replace(/\\"/g, '')
-    : '["https://topcoder-dev.auth0.com/", "https://api.topcoder.com"]',
+    : '["https://api.topcoder-dev.com", "https://api.topcoder.com", "https://topcoder-dev.auth0.com/"]',
 
   PAGE_SIZE: process.env.PAGE_SIZE || 20,
   MAX_PAGE_SIZE: parseInt(process.env.MAX_PAGE_SIZE) || 100,

config/test.js

@@ -0,0 +1,40 @@
+/**
+ * The configuration file.
+ */
+
+module.exports = {
+  BUSAPI_URL: 'http://localhost:9000',
+  WAIT_TIME: process.env.WAIT_TIME || 5000,
+  AUTH_V2_URL: process.env.AUTH_V2_URL || 'https://topcoder-dev.auth0.com/oauth/ro',
+  AUTH_V2_CLIENT_ID: process.env.AUTH_V2_CLIENT_ID || '',
+  AUTH_V3_URL: process.env.AUTH_V3_URL || 'https://api.topcoder-dev.com/v3/authorizations',
+  ADMIN_CREDENTIALS_USERNAME: process.env.ADMIN_CREDENTIALS_USERNAME || '',
+  ADMIN_CREDENTIALS_PASSWORD: process.env.ADMIN_CREDENTIALS_PASSWORD || '',
+  USER_CREDENTIALS_USERNAME: process.env.USER_CREDENTIALS_USERNAME || '',
+  USER_CREDENTIALS_PASSWORD: process.env.USER_CREDENTIALS_PASSWORD || '',
+  MANAGER_CREDENTIALS_USERNAME: process.env.MANAGER_CREDENTIALS_USERNAME || '',
+  MANAGER_CREDENTIALS_PASSWORD: process.env.MANAGER_CREDENTIALS_PASSWORD || '',
+  COPILOT_CREDENTIALS_USERNAME: process.env.COPILOT_CREDENTIALS_USERNAME || '',
+  COPILOT_CREDENTIALS_PASSWORD: process.env.COPILOT_CREDENTIALS_PASSWORD || '',
+  AUTOMATED_TESTING_REPORTERS_FORMAT: process.env.AUTOMATED_TESTING_REPORTERS_FORMAT
+    ? process.env.AUTOMATED_TESTING_REPORTERS_FORMAT.split(',')
+    : ['cli', 'html'],
+  USER_ID_BY_ADMIN: '0bcb0d86-09bb-410a-b2b1-fba90d1a7699',
+  USER_ID_BY_TESTER: 'bdcb113f-6715-40fd-8dab-14aa01327ae9',
+  PROVIDER_ID_BY_ADMIN: '7637ae1a-3b7c-44eb-a5ed-10ea02f1885d',
+  PROVIDER_ID_BY_TESTER: '4df1f70b-52e8-4e55-b79b-15ffc2c6a4f6',
+  SKILL_ID_BY_ADMIN: '0aec2956-cbcb-4c80-8c00-25cc02a71611',
+  SKILL_ID_BY_TESTER: '9ad4eb94-4029-49ff-b01a-9ac0a964dd35',
+  ROLE_ID_BY_ADMIN: '8607ddb3-abf6-4512-a618-c60d4771174b',
+  ROLE_ID_BY_TESTER: '58fffc97-db2a-48a7-9066-cfc83bbde7b5',
+  ORGANIZATION_ID_BY_ADMIN: 'a866fd95-c2d5-4fa8-b341-a367138c9911',
+  ORGANIZATION_ID_BY_TESTER: 'f1e7cf4d-e688-4037-980e-820f12c822d9',
+  ACHIEVEMENTS_PROVIDER_ID_BY_ADMIN: 'b1b22eba-ba1b-4740-8f70-da607de52ce7',
+  ACHIEVEMENTS_PROVIDER_ID_BY_TESTER: '1ad72e52-4b44-4634-9c02-4e792c156b43',
+  ATTRIBUTE_GROUP_ID_BY_ADMIN: '03df695f-0b97-4fe3-ae29-17f1aa0addc4',
+  ATTRIBUTE_GROUP_ID_BY_TESTER: '8599dbcb-7301-46a1-ab23-7b3dfd610f26',
+  ATTRIBUTE_ID_BY_ADMIN: '5c22a53f-6530-4f92-b934-64cc9b05f1d0',
+  ATTRIBUTE_ID_BY_TESTER: 'a15f160c-5634-4af5-9967-cc4b28bdc94c',
+  ACHIEVEMENT_ID_BY_ADMIN: '9a771c88-1e57-4a47-bf20-a9a62c118237',
+  ACHIEVEMENT_ID_BY_TESTER: '6499865c-6cb6-49c1-991f-be95fb03ed56'
+}