Skip to content

Commit 89e3348

Browse files
ajeftsajefts
authored
Merge pull request #56 from appirio-tech/dev
promote to prod
2 parents b56a58d + 723d140 commit 89e3348

40 files changed

+4067
-1094
lines changed

‎README.md‎

100644100755
Lines changed: 143 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -1,49 +1,158 @@
11
# Review Microservice
2-
A microservice for generic units of work (e.g., a challenge)
2+
Microservice for supporting Topcoder Online Review platform.
33

4-
## Getting Started
5-
This microservice is using [dropwizard](dropwizard.io) as the REST API framework and implements the [v3 API spec](https://docs.google.com/a/appirio.com/presentation/d/15pucEI0MHj9y778EyaAWGh4MBH-I73i1-GS0ir7FhxE/edit#slide=id.g29c3ffcc3_020). If you are unfamiliar with dropwizard, take a look at their [Getting Started](http://dropwizard.io/getting-started.html) page.
4+
# Prerequisities
65

7-
### Building
8-
We are using Maven 2 to build and building is as simple as `mvn package`. *Make sure you are using Java 8 for build and runtime*
6+
- Java 8 with Maven 2+
7+
- Docker
98

10-
### Configuration
11-
Configuration is managed through a yaml file. The `src/main/resources/review-service.yaml` file should contain the necessary configuration for you to run locally.
9+
# Local Deployment
1210

13-
### Framework Stack
14-
- dropwizard-core - core REST API
15-
- dropwizard-auth - user authentication
16-
- dropwizard-jdbi - database access
17-
- dropwizard-testing - unit testing
18-
- mysql-connector-java - mysql JDBC driver
19-
- fest-assert-core, mockito-core - unit testing
20-
- slf4j - logging
11+
### Start Informix instance via Docker
2112

22-
### Database
23-
We are using mysql RDS for the database. For development use, connect to the topcoder-dev instance. Database access should be limited to the DAO layer. We are using the [JDBI](http://jdbi.org/) framework for database access. Where possible we should use the [SQL Objects API](http://jdbi.org/sql_object_overview/) approach listed in the JDBI documentation.
13+
- Set an IP for Docker by setting the environment variable `DOCKER_IP`. Recommended to set to `0.0.0.0`
2414

25-
For this service, the DAO is managed by `ReviewDAO`
15+
- Command to set the environment variable `DOCKER_IP`
2616

27-
**We need to keep the data model clean and name properties of our representation (model) objects as the column names so we can use the automatic bean mapping facilities instead of needing to manually wire things together. Any deviation requires prior approval.**
17+
```
18+
export DOCKER_IP=0.0.0.0
19+
```
20+
21+
- To run Informix locally, go to the `local` folder and execute the following command:
22+
23+
```
24+
docker-compose up
25+
```
2826

29-
### API Endpoint
30-
The API endpoint for this service is `com.appirio.service.review.resources.ReviewResource` It contains the specific resource paths supported by this service. The root of the service is /v3/reviews. API methods should be annotated with the appropriate jax-rs annotations and the `@Timed` annotation for metrics. An example method is shown below.
27+
If you should ever encounter the error:
28+
29+
```
30+
[java] Caused by: com.topcoder.db.connectionfactory.DBConnectionException: error occurs while creating the connection.
31+
You already have 20 connections at this
32+
time. Any further connections
3133
```
32-
@GET
33-
@Timed
34-
public ApiResponse getReviews(@Auth AuthUser auth,
35-
@APIQueryParam(repClass = Review.class) QueryParameter queryParameter)
36-
logger.debug("getReviews, filter : " + queryParameter.getFilter().getFields());
3734

38-
return MetadataApiResponseFactory.createResponse(reviewManager.getReviews(auth, queryParameter,
39-
new BaseAuthorizationContext(auth)));
40-
}
35+
Try to restart the informix database server as below:
36+
```
37+
#### log into the container based on the container name got in the previous step
38+
docker exec -it local_informix bash
39+
40+
#### restart the database server
41+
onmode -ky
42+
oninit
43+
```
44+
45+
### Inserting test records
46+
47+
- Test data is present in SQL file (`test_data.sql`) in `local` folder.
48+
49+
- Transfer the `test_data.sql` to docker environment by executing the following command
50+
4151
```
52+
docker cp test_data.sql local_informix:/home/informix
53+
```
54+
55+
- Insert the records in `tcs_catalog` database by executing the following command
56+
57+
```
58+
docker exec -it local_informix dbaccess tcs_catalog test_data.sql
59+
```
60+
61+
- Docker environment is ready to be used in our service.
62+
63+
### Configuration Variables
64+
The configuration file `service/src/main/resources/review-service.yaml` contain the necessary configuration to run the microservice locally. Below configuration variables need to be set using environment variables.
65+
66+
|Name | Description |
67+
|--------------------|:-----------------------------------------:|
68+
|AUTH_DOMAIN | Authentication domain of JWT, Dont set unless you have valid tokens with given AUTH_DOMAIN, Defaulted to 'topcoder-dev.com' |
69+
|IP | The IP of your docker container |
70+
|OLTP_USER | The Informix OLTP user |
71+
|OLTP_PW | The Informix OLTP user password |
72+
|OLTP_URL | The JDBC URL of Informix OLTP database |
73+
|DW_USER | The Informix DW user |
74+
|DW_PW | The Informix DW user password |
75+
|DW_URL | The JDBC URL of Informix DW database |
76+
|TC_JWT_KEY | Secret used in JWT Service, Used for signature verification as well |
77+
78+
In Linux, Set the environment variables as
79+
80+
```
81+
export AUTH_DOMAIN=...
82+
export IP=...
83+
export OLTP_USER=...
84+
export OLTP_PW=...
85+
export OLTP_URL=...
86+
export DW_USER=...
87+
export DW_PW=...
88+
export DW_URL=...
89+
```
90+
91+
### Working credentials for local deployment
92+
93+
```
94+
export IP=0.0.0.0
95+
export OLTP_USER=informix
96+
export OLTP_PW=1nf0rm1x
97+
export OLTP_URL=jdbc:informix-sqli://$IP:2021/tcs_catalog:INFORMIXSERVER=informixoltp_tcp;IFX_LOCK_MODE_WAIT=5;OPTCOMPIND=0;STMT_CACHE=1
98+
export DW_USER=informix
99+
export DW_PW=1nf0rm1x
100+
export DW_URL=jdbc:informix-sqli://$IP:2021/common_dw:INFORMIXSERVER=informixoltp_tcp;IFX_LOCK_MODE_WAIT=5;OPTCOMPIND=0;STMT_CACHE=1
101+
export TC_JWT_KEY=secret
102+
```
103+
104+
### Set up Local Maven repository for Appirio Maven
105+
106+
- Since maven.appirio.net is not accessible, We need to set up a local maven repository with the zip file `temp-maven-repo-master.zip`
107+
108+
- Unzip the contents to any of the directory in your System. **In Linux, the directory should not be a mounted directory like pen drive, hard disk, etc..**
109+
110+
- Open `pom.xml` in `service` directory and replace the `<url>` of Local repository with `absolute path of the unzipped file`. E.g. If you have unzipped the files to `/home/user/temp-maven-repo-master`, then POM entry will look like
111+
112+
```
113+
<repository>
114+
<id>1_temp_appirio_maven_repo</id>
115+
<name>Appirio Maven Local Repository</name>
116+
<url>file:///home/user/temp-maven-repo-master</url>
117+
</repository>
118+
```
119+
120+
### Build Microservice
121+
122+
In the `service` folder of this project, run the following command to build the service.
123+
124+
```
125+
mvn clean package
126+
```
127+
128+
### Start Microservice
129+
130+
- Before starting please ensure that all environment variables listed above is set properly and docker is up and running.
131+
132+
- Then start the service using the following command in the 'service' folder:
133+
134+
```
135+
java -jar ./target/review-microservice-*.jar server ./src/main/resources/review-service.yaml
136+
```
137+
138+
### Verification
139+
140+
Though there are many endpoints, For starters, it's sufficient to verify two end points.
141+
142+
- Endpoint `/v3/scorecards` doesn't require Authentication
143+
144+
- Endpoint `/v3/reviews` require Authentication
145+
146+
Use the postman collection in `Postman` directory to verify the end points.
147+
148+
- **Ensure to import the postman environment before testing the endpoints**
149+
150+
- Request `Get list of all Scorecards` returns all scorecard templates present in the database
151+
152+
**Loaded test data has details for 3 submissions for a single competition reviewed by 2 reviewers**
42153

43-
Note the `TCAuth` parameter provided by the `ServiceAuthenticator` class will provide user auth information and is a required parameterfor all secured API calls.
44-
You must set up `TC_JWT_KEY` environment variable, the sample value is `4WvoZLWhFPZ5jauw2+XCU+p772S4oBN25tNPyjHR`.
154+
- Request `Get list of all reviews Reviewer1` return the reviews done by Reviewer 1.
45155

46-
### Testing
47-
All API endpoint methods should have unit tests. For this service we are following the dropwizard testing recommendations as described [here](http://dropwizard.io/manual/testing.html)
156+
- Request `Get list of all reviews Reviewer2` return the reviews done by Reviewer 2.
48157

49-
For manual API testing, the [postman chrome app](https://chrome.google.com/webstore/detail/postman-rest-client/fdmmgilgnpjigdojojpjoooidkmcomcm?hl=en) is useful.
158+
**Note: Some of the fields in returned output will be NULL, It's expected. That's how the internal controllers and services are written**

‎local/docker-compose.yml‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,15 @@
11
version: '2'
22
services:
33
informix:
4+
container_name: local_informix
45
image: "appiriodevops/informix:1b3d4ef"
56
ports:
67
- "2021:2021"
78
kafka:
89
image: spotify/kafka
910
environment:
10-
ADVERTISED_HOST: $DOCKER_IP
11-
ADVERTISED_PORT: 9092
11+
ADVERTISED_HOST: ${DOCKER_IP}
12+
ADVERTISED_PORT: 2181
1213
ports:
1314
- "2181:2181"
1415
- "9092:9092"

‎local/run.sh‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
export IP=192.168.1.2
2+
export OLTP_USER=informix
3+
export OLTP_PW=1nf0rm1x
4+
export OLTP_URL=jdbc:informix-sqli://$IP:2021/tcs_catalog:INFORMIXSERVER=informixoltp_tcp;IFX_LOCK_MODE_WAIT=5;OPTCOMPIND=0;STMT_CACHE=1
5+
export DW_USER=informix
6+
export DW_PW=1nf0rm1x
7+
export DW_URL=jdbc:informix-sqli://$IP:2021/common_dw:INFORMIXSERVER=informixoltp_tcp;IFX_LOCK_MODE_WAIT=5;OPTCOMPIND=0;STMT_CACHE=1
8+
export TC_JWT_KEY=secret
9+
10+
11+
java -jar ../service/target/review-microservice-*.jar server ../service/src/main/resources/review-service.yaml

‎local/test_data.sql‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
database tcs_catalog;
2+
3+
/* Data for a project*/
4+
5+
insert into project(project_id, project_status_id, project_category_id, create_user, create_date, modify_user, modify_date)
6+
values(1, 1, 1, 132456, current, 132456, current);
7+
8+
/* Data for 2 reviewers and 3 submitters based on the token provided */
9+
insert into resource(resource_id, resource_role_id, project_id, project_phase_id, user_id, create_user, create_date, modify_user, modify_date) values(2, 4, 1, 13, 1234, 132456, current, 132456, current);
10+
11+
insert into resource(resource_id, resource_role_id, project_id, project_phase_id, user_id, create_user, create_date, modify_user, modify_date) values(3, 4, 1, 13, 1235, 132456, current, 132456, current);
12+
13+
insert into resource(resource_id, resource_role_id, project_id, project_phase_id, user_id, create_user, create_date, modify_user, modify_date) values(4, 1, 1, 13, 1200, 132456, current, 132456, current);
14+
15+
insert into resource(resource_id, resource_role_id, project_id, project_phase_id, user_id, create_user, create_date, modify_user, modify_date) values(5, 1, 1, 13, 1201, 132456, current, 132456, current);
16+
17+
insert into resource(resource_id, resource_role_id, project_id, project_phase_id, user_id, create_user, create_date, modify_user, modify_date) values(6, 1, 1, 13, 1202, 132456, current, 132456, current);
18+
19+
/* Data for 3 uploads */
20+
21+
insert into upload(upload_id, project_id, project_phase_id, resource_id, upload_type_id, upload_status_id, parameter, create_user, create_date, modify_user, modify_date) values(1, 1, 12, 4, 1, 1, 'someparam', 132456, current, 132456, current);
22+
23+
insert into upload(upload_id, project_id, project_phase_id, resource_id, upload_type_id, upload_status_id, parameter, create_user, create_date, modify_user, modify_date) values(2, 1, 12, 5, 1, 1, 'someparam', 132456, current, 132456, current);
24+
25+
insert into upload(upload_id, project_id, project_phase_id, resource_id, upload_type_id, upload_status_id, parameter, create_user, create_date, modify_user, modify_date) values(3, 1, 12, 6, 1, 1, 'someparam', 132456, current, 132456, current);
26+
27+
28+
/* Data for 3 submission */
29+
30+
insert into submission(submission_id, upload_id, submission_status_id, submission_type_id, create_user, create_date, modify_user, modify_date)
31+
values(1, 1, 1, 1, 132456, current, 132456, current);
32+
33+
insert into submission(submission_id, upload_id, submission_status_id, submission_type_id, create_user, create_date, modify_user, modify_date)
34+
values(2, 2, 1, 1, 132456, current, 132456, current);
35+
36+
insert into submission(submission_id, upload_id, submission_status_id, submission_type_id, create_user, create_date, modify_user, modify_date)
37+
values(3, 3, 1, 1, 132456, current, 132456, current);
38+
39+
40+
/* 3 submissions for a single competition reviewed by 2 reviewers*/
41+
42+
insert into review(review_id, resource_id, submission_id, project_phase_id, scorecard_id, committed, score, initial_score, create_user, create_date, modify_user, modify_date)
43+
values(1, 2, 1, 13, 30000410, 1, 100, 95, 132456, current, 132456, current);
44+
45+
insert into review(review_id, resource_id, submission_id, project_phase_id, scorecard_id, committed, score, initial_score, create_user, create_date, modify_user, modify_date)
46+
values(2, 3, 1, 13, 30000410, 1, 100, 95, 132456, current, 132456, current);
47+
48+
insert into review(review_id, resource_id, submission_id, project_phase_id, scorecard_id, committed, score, initial_score, create_user, create_date, modify_user, modify_date)
49+
values(3, 2, 2, 13, 30000410, 1, 65, 55, 132456, current, 132456, current);
50+
51+
insert into review(review_id, resource_id, submission_id, project_phase_id, scorecard_id, committed, score, initial_score, create_user, create_date, modify_user, modify_date)
52+
values(4, 3, 2, 13, 30000410, 1, 70, 60, 132456, current, 132456, current);
53+
54+
insert into review(review_id, resource_id, submission_id, project_phase_id, scorecard_id, committed, score, initial_score, create_user, create_date, modify_user, modify_date)
55+
values(5, 2, 3, 13, 30000410, 1, 90, 87, 132456, current, 132456, current);
56+
57+
insert into review(review_id, resource_id, submission_id, project_phase_id, scorecard_id, committed, score, initial_score, create_user, create_date, modify_user, modify_date)
58+
values(6, 3, 3, 13, 30000410, 1, 84.36, 78, 132456, current, 132456, current);

0 commit comments

Comments
 (0)