Skip to content

Commit c2777e6

Browse files
jashmoreAtlassianjashmoreAtlassian
committed
NONE: Added core implemetation for the core sqs listeners
1 parent d7ea7c3 commit c2777e6

File tree

86 files changed

+6816
-2
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

86 files changed

+6816
-2
lines changed

‎.gitignore

+5
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
# Compiled class file
22
*.class
3+
target/
34

45
# Log file
56
*.log
@@ -21,3 +22,7 @@
2122

2223
# virtual machine crash logs, see http://www.java.com/en/download/help/error_hotspot.xml
2324
hs_err_pid*
25+
26+
# IntelliJ
27+
.idea
28+
*.iml

‎.travis.yml

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
language: java

‎README.md

+76-2
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,76 @@
1-
# java-dynamic-sqs-listener
2-
Sets up a SQS Listener that can be dynamically configured while it is executing.
1+
[![Build Status](https://travis-ci.org/JaidenAshmore/java-dynamic-sqs-listener.png)](https://travis-ci.org/JaidenAshmore/java-dynamic-sqs-listener)
2+
[![Coverage Status](https://coveralls.io/repos/github/JaidenAshmore/java-dynamic-sqs-listener/badge.svg?branch=master)](https://coveralls.io/github/JaidenAshmore/java-dynamic-sqs-listener?branch=master)
3+
4+
# Java Dynamic SQS Listener - SQS Listener with easy configuration and extensibility
5+
SQS Listener implementation that is designed from the ground up to allow for easier configuration and extensibility.
6+
7+
## Why Dynamic SQS Listener?
8+
Whilst the other implementations of SQS listeners were easy to set up and provided most use cases, they don't provide the extensibility and dynamic
9+
requirements that are needed by some services. Therefore, the following characteristics
10+
were crucial components in the design of this implementation:
11+
12+
- **Configurability**: The Dynamic SQS Listener is built with configurability and extensibility in mind allowing for different parts
13+
of the listener to be replaced or extended based on the specific use case. For example, you may find that the specific
14+
implementation for how messages are consumed from the SQS queue doesn't work as intended and so can be replaced without
15+
needing to make changes to the other parts of the framework.
16+
- **Dynamic control**: The other major use case for this implementation is allowing for the configuration of the listener while it is running. For example,
17+
your service is rolling out a new feature and wants to place the rate of message listening behind a feature flag. With the use of a
18+
`ConcurrentMessageBroker` the service is able to dynamically throttle the number of threads processing messages via a
19+
simple properties supplier.
20+
21+
## Usage
22+
23+
### API
24+
This modules contains the API for the framework and includes a lot of documentation about how each component should be
25+
interacting with each other. If this framework is being extended by adding a different time of message retriever, the API
26+
is the only dependency that would be needed.
27+
28+
### Core
29+
This provides basic Java implementations of the API which should cover multiple use cases. There should be no dependencies
30+
on any external dependencies to promote better integration by reducing dependency hell. Because of this setting up all
31+
the components needed for the framework can take a bit more boilerplate code for the consumer but the advantage is
32+
the improved ease of integration.
33+
34+
### Spring Boot
35+
Spring Starter used to reduce the amount of boilerplate code by providing auto configuration and annotations for hooking
36+
into the framework. This still needs to be implemented.
37+
38+
### Other implementations?
39+
Pull Requests are welcome for adding any other implementations for frameworks or types of listeners. For example, you want to try to use RxJava for your
40+
`MessageBroker` implementation you could add this in the core package for others to use.
41+
42+
## Architecture
43+
As can be seen in [java-dynamic-sqs-listener-api](./java-dynamic-sqs-listener-api) the listener has been divided into
44+
four main responsibilities. Each of these have been built with no reliance on each other and therefore the consumer
45+
can use or interchange the implementations of any of these with ease.
46+
47+
### [Message Retriever](./java-dynamic-sqs-listener-api/src/main/java/com/jashmore/sqs/retriever)
48+
The message retriever is one of the most important parts of the architecture and it has the responsibility of figuring
49+
out how to obtain new messages from the queue. The [MessageRetriever](./java-dynamic-sqs-listener-api/src/main/java/com/jashmore/sqs/retriever/MessageRetriever.java)
50+
and [AsyncMessageRetriever](./java-dynamic-sqs-listener-api/src/main/java/com/jashmore/sqs/retriever/AsyncMessageRetriever.java)
51+
API has been kept to a bare minimum and so the complexity for how it wants to handle obtaining the messages is kept
52+
internal to the implementation. Note that the `MessageRetriever` must be thread safe as there could be multiple threads
53+
all requesting messages to process.
54+
55+
### [Message Processor](./java-dynamic-sqs-listener-api/src/main/java/com/jashmore/sqs/processor)
56+
The [MessageProcessor](./java-dynamic-sqs-listener-api/src/main/java/com/jashmore/sqs/processor/MessageProcessor.java)
57+
is the part that handles how it should process the message that has been obtained from the SQS queue. It will take the
58+
message and run it through a Java method that should be able to handle this message this message. Through this process
59+
it will extra all of the arguments to be passed into the method via argument resolution.
60+
61+
### [Argument Resolvers](./java-dynamic-sqs-listener-api/src/main/java/com/jashmore/sqs/argument)
62+
The message processor above needs a way to know how arguments in the method should be populated. For example, if the
63+
method wants to know the message id for this message how does this get populated. This is the responsibility of the
64+
[ArgumentResolverService](./java-dynamic-sqs-listener-api/src/main/java/com/jashmore/sqs/argument/ArgumentResolverService.java)
65+
where it will be able to look at the method parameters and resolve the arguments from this message. To increase
66+
testability and extensibility the [ArgumentResolver](./java-dynamic-sqs-listener-api/src/main/java/com/jashmore/sqs/argument/ArgumentResolver.java)
67+
is used to handle each different type of argument, for example the
68+
[PayloadArgumentResolver](./java-dynamic-sqs-listener-core/src/main/java/com/jashmore/sqs/argument/payload/PayloadArgumentResolver.java)
69+
which populates arguments annotated with [@Payload](./java-dynamic-sqs-listener-core/src/main/java/com/jashmore/sqs/argument/payload/Payload.java)
70+
by parsing the message body into that argument.
71+
72+
### [Message Broker](./java-dynamic-sqs-listener-api/src/main/java/com/jashmore/sqs/broker)
73+
This is the main container for listening to messages on a queue and applying it to the specific message consumer. It is
74+
basically the high level broker that glues all of the above components together. For example, the
75+
[ConcurrentMessageBroker](./java-dynamic-sqs-listener-core/src/main/java/com/jashmore/sqs/broker/concurrent/ConcurrentMessageBroker.java)
76+
will spin up multiple threads that will retrieve messages and pass them to the message processor for processing.

‎configuration/checkstyle/MODIFICATIONS.md

+7
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
### Changes to Google Checkstyle
2+
As some of the rules I don't agree with and can be frustrating to use I have done some slight modifications to the sheet. The changes that I
3+
have made are:
4+
5+
- Line length increased to 160 from 100
6+
- Tab width doubled, e.g. from 2 spaces to 4 spaces
7+
- Removed the requirement for constructor JavaDoc by adding JavadocMethod.tokens without CTOR_DEF

‎configuration/checkstyle/google_6_18_checkstyle.xml

+213
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,213 @@
1+
<?xml version="1.0"?>
2+
<!DOCTYPE module PUBLIC
3+
"-//Puppy Crawl//DTD Check Configuration 1.3//EN"
4+
"http://www.puppycrawl.com/dtds/configuration_1_3.dtd">
5+
6+
<!--
7+
Checkstyle configuration that checks the Google coding conventions from Google Java Style
8+
that can be found at https://google.github.io/styleguide/javaguide.html.
9+
Checkstyle is very configurable. Be sure to read the documentation at
10+
http://checkstyle.sf.net (or in your downloaded distribution).
11+
To completely disable a check, just comment it out or delete it from the file.
12+
Authors: Max Vetrenko, Ruslan Diachenko, Roman Ivanov.
13+
-->
14+
15+
<module name="Checker">
16+
<property name="charset" value="UTF-8"/>
17+
18+
<property name="severity" value="warning"/>
19+
20+
<property name="fileExtensions" value="java, properties, xml"/>
21+
<!-- Checks for whitespace -->
22+
<!-- See http://checkstyle.sf.net/config_whitespace.html -->
23+
<module name="FileTabCharacter">
24+
<property name="eachLine" value="true"/>
25+
</module>
26+
27+
<module name="TreeWalker">
28+
<module name="OuterTypeFilename"/>
29+
<module name="IllegalTokenText">
30+
<property name="tokens" value="STRING_LITERAL, CHAR_LITERAL"/>
31+
<property name="format"
32+
value="\\u00(08|09|0(a|A)|0(c|C)|0(d|D)|22|27|5(C|c))|\\(0(10|11|12|14|15|42|47)|134)"/>
33+
<property name="message" value="Avoid using corresponding octal or Unicode escape."/>
34+
</module>
35+
<module name="AvoidEscapedUnicodeCharacters">
36+
<property name="allowEscapesForControlCharacters" value="true"/>
37+
<property name="allowByTailComment" value="true"/>
38+
<property name="allowNonPrintableEscapes" value="true"/>
39+
</module>
40+
<module name="LineLength">
41+
<property name="max" value="160"/>
42+
<property name="ignorePattern" value="^package.*|^import.*|a href|href|http://|https://|ftp://"/>
43+
</module>
44+
<module name="AvoidStarImport"/>
45+
<module name="OneTopLevelClass"/>
46+
<module name="NoLineWrap"/>
47+
<module name="EmptyBlock">
48+
<property name="option" value="TEXT"/>
49+
<property name="tokens" value="LITERAL_TRY, LITERAL_FINALLY, LITERAL_IF, LITERAL_ELSE, LITERAL_SWITCH"/>
50+
</module>
51+
<module name="NeedBraces"/>
52+
<module name="LeftCurly">
53+
<property name="maxLineLength" value="160"/>
54+
</module>
55+
<module name="RightCurly"/>
56+
<module name="RightCurly">
57+
<property name="option" value="alone"/>
58+
<property name="tokens"
59+
value="CLASS_DEF, METHOD_DEF, CTOR_DEF, LITERAL_FOR, LITERAL_WHILE, LITERAL_DO, STATIC_INIT, INSTANCE_INIT"/>
60+
</module>
61+
<module name="WhitespaceAround">
62+
<property name="allowEmptyConstructors" value="true"/>
63+
<property name="allowEmptyMethods" value="true"/>
64+
<property name="allowEmptyTypes" value="true"/>
65+
<property name="allowEmptyLoops" value="true"/>
66+
<message key="ws.notFollowed"
67+
value="WhitespaceAround: ''{0}'' is not followed by whitespace. Empty blocks may only be represented as '{}' when not part of a multi-block statement (4.1.3)"/>
68+
<message key="ws.notPreceded"
69+
value="WhitespaceAround: ''{0}'' is not preceded with whitespace."/>
70+
</module>
71+
<module name="OneStatementPerLine"/>
72+
<module name="MultipleVariableDeclarations"/>
73+
<module name="ArrayTypeStyle"/>
74+
<module name="MissingSwitchDefault"/>
75+
<module name="FallThrough"/>
76+
<module name="UpperEll"/>
77+
<module name="ModifierOrder"/>
78+
<module name="EmptyLineSeparator">
79+
<property name="allowNoEmptyLineBetweenFields" value="true"/>
80+
</module>
81+
<module name="SeparatorWrap">
82+
<property name="tokens" value="DOT"/>
83+
<property name="option" value="nl"/>
84+
</module>
85+
<module name="SeparatorWrap">
86+
<property name="tokens" value="COMMA"/>
87+
<property name="option" value="EOL"/>
88+
</module>
89+
<module name="PackageName">
90+
<property name="format" value="^[a-z]+(\.[a-z][a-z0-9]*)*$"/>
91+
<message key="name.invalidPattern"
92+
value="Package name ''{0}'' must match pattern ''{1}''."/>
93+
</module>
94+
<module name="TypeName">
95+
<message key="name.invalidPattern"
96+
value="Type name ''{0}'' must match pattern ''{1}''."/>
97+
</module>
98+
<module name="MemberName">
99+
<property name="format" value="^[a-z][a-z0-9][a-zA-Z0-9]*$"/>
100+
<message key="name.invalidPattern"
101+
value="Member name ''{0}'' must match pattern ''{1}''."/>
102+
</module>
103+
<module name="ParameterName">
104+
<property name="format" value="^[a-z][a-z0-9][a-zA-Z0-9]*$"/>
105+
<message key="name.invalidPattern"
106+
value="Parameter name ''{0}'' must match pattern ''{1}''."/>
107+
</module>
108+
<module name="CatchParameterName">
109+
<property name="format" value="^[a-z][a-z0-9][a-zA-Z0-9]*$"/>
110+
<message key="name.invalidPattern"
111+
value="Catch parameter name ''{0}'' must match pattern ''{1}''."/>
112+
</module>
113+
<module name="LocalVariableName">
114+
<property name="tokens" value="VARIABLE_DEF"/>
115+
<property name="format" value="^[a-z][a-z0-9][a-zA-Z0-9]*$"/>
116+
<property name="allowOneCharVarInForLoop" value="true"/>
117+
<message key="name.invalidPattern"
118+
value="Local variable name ''{0}'' must match pattern ''{1}''."/>
119+
</module>
120+
<module name="ClassTypeParameterName">
121+
<property name="format" value="(^[A-Z][0-9]?)$|([A-Z][a-zA-Z0-9]*[T]$)"/>
122+
<message key="name.invalidPattern"
123+
value="Class type name ''{0}'' must match pattern ''{1}''."/>
124+
</module>
125+
<module name="MethodTypeParameterName">
126+
<property name="format" value="(^[A-Z][0-9]?)$|([A-Z][a-zA-Z0-9]*[T]$)"/>
127+
<message key="name.invalidPattern"
128+
value="Method type name ''{0}'' must match pattern ''{1}''."/>
129+
</module>
130+
<module name="InterfaceTypeParameterName">
131+
<property name="format" value="(^[A-Z][0-9]?)$|([A-Z][a-zA-Z0-9]*[T]$)"/>
132+
<message key="name.invalidPattern"
133+
value="Interface type name ''{0}'' must match pattern ''{1}''."/>
134+
</module>
135+
<module name="NoFinalizer"/>
136+
<module name="GenericWhitespace">
137+
<message key="ws.followed"
138+
value="GenericWhitespace ''{0}'' is followed by whitespace."/>
139+
<message key="ws.preceded"
140+
value="GenericWhitespace ''{0}'' is preceded with whitespace."/>
141+
<message key="ws.illegalFollow"
142+
value="GenericWhitespace ''{0}'' should followed by whitespace."/>
143+
<message key="ws.notPreceded"
144+
value="GenericWhitespace ''{0}'' is not preceded with whitespace."/>
145+
</module>
146+
<module name="Indentation">
147+
<property name="basicOffset" value="4"/>
148+
<property name="braceAdjustment" value="0"/>
149+
<property name="caseIndent" value="4"/>
150+
<property name="throwsIndent" value="8"/>
151+
<property name="lineWrappingIndentation" value="8"/>
152+
<property name="arrayInitIndent" value="4"/>
153+
</module>
154+
<module name="AbbreviationAsWordInName">
155+
<property name="ignoreFinal" value="false"/>
156+
<property name="allowedAbbreviationLength" value="1"/>
157+
</module>
158+
<module name="OverloadMethodsDeclarationOrder"/>
159+
<module name="VariableDeclarationUsageDistance"/>
160+
<module name="CustomImportOrder">
161+
<property name="specialImportsRegExp" value="com.google"/>
162+
<property name="sortImportsInGroupAlphabetically" value="true"/>
163+
<property name="customImportOrderRules"
164+
value="STATIC###SPECIAL_IMPORTS###THIRD_PARTY_PACKAGE###STANDARD_JAVA_PACKAGE"/>
165+
</module>
166+
<module name="MethodParamPad"/>
167+
<module name="OperatorWrap">
168+
<property name="option" value="NL"/>
169+
<property name="tokens"
170+
value="BAND, BOR, BSR, BXOR, DIV, EQUAL, GE, GT, LAND, LE, LITERAL_INSTANCEOF, LOR, LT, MINUS, MOD, NOT_EQUAL, PLUS, QUESTION, SL, SR, STAR "/>
171+
</module>
172+
<module name="AnnotationLocation">
173+
<property name="tokens" value="CLASS_DEF, INTERFACE_DEF, ENUM_DEF, METHOD_DEF, CTOR_DEF"/>
174+
</module>
175+
<module name="AnnotationLocation">
176+
<property name="tokens" value="VARIABLE_DEF"/>
177+
<property name="allowSamelineMultipleAnnotations" value="true"/>
178+
</module>
179+
<module name="NonEmptyAtclauseDescription"/>
180+
<module name="JavadocTagContinuationIndentation"/>
181+
<module name="SummaryJavadoc">
182+
<property name="forbiddenSummaryFragments"
183+
value="^@return the *|^This method returns |^A [{]@code [a-zA-Z0-9]+[}]( is a )"/>
184+
</module>
185+
<module name="JavadocParagraph"/>
186+
<module name="AtclauseOrder">
187+
<property name="tagOrder" value="@param, @return, @throws, @deprecated"/>
188+
<property name="target" value="CLASS_DEF, INTERFACE_DEF, ENUM_DEF, METHOD_DEF, CTOR_DEF, VARIABLE_DEF"/>
189+
</module>
190+
<module name="JavadocMethod">
191+
<property name="scope" value="public"/>
192+
<property name="allowMissingParamTags" value="true"/>
193+
<property name="allowMissingThrowsTags" value="true"/>
194+
<property name="allowMissingReturnTag" value="true"/>
195+
<property name="minLineCount" value="2"/>
196+
<property name="allowedAnnotations" value="Override, Test"/>
197+
<property name="allowThrowsTagsForSubclasses" value="true"/>
198+
<property name="tokens" value="METHOD_DEF,ANNOTATION_FIELD_DEF"/>
199+
</module>
200+
<module name="MethodName">
201+
<property name="format" value="^[a-z][a-z0-9][a-zA-Z0-9_]*$"/>
202+
<message key="name.invalidPattern"
203+
value="Method name ''{0}'' must match pattern ''{1}''."/>
204+
</module>
205+
<module name="SingleLineJavadoc">
206+
<property name="ignoreInlineTags" value="false"/>
207+
</module>
208+
<module name="EmptyCatchBlock">
209+
<property name="exceptionVariableName" value="expected"/>
210+
</module>
211+
<module name="CommentsIndentation"/>
212+
</module>
213+
</module>

‎examples/java-dynamic-sqs-listener-core-examples/pom.xml

+46
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
<?xml version="1.0" encoding="UTF-8"?>
2+
<project xmlns="http://maven.apache.org/POM/4.0.0"
3+
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
4+
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
5+
<parent>
6+
<artifactId>examples</artifactId>
7+
<groupId>com.jashmore</groupId>
8+
<version>0.0.1-SNAPSHOT</version>
9+
</parent>
10+
<modelVersion>4.0.0</modelVersion>
11+
12+
<artifactId>java-dynamic-sqs-listener-core-examples</artifactId>
13+
14+
<name>Java Dynamic SQS Listener - Core - Examples</name>
15+
<description>Contains examples for using the core implementation of the listener with plan Java objects.</description>
16+
17+
<dependencies>
18+
<dependency>
19+
<groupId>com.amazonaws</groupId>
20+
<artifactId>aws-java-sdk-sqs</artifactId>
21+
</dependency>
22+
23+
<dependency>
24+
<groupId>org.projectlombok</groupId>
25+
<artifactId>lombok</artifactId>
26+
<scope>provided</scope>
27+
</dependency>
28+
29+
<dependency>
30+
<groupId>com.fasterxml.jackson.core</groupId>
31+
<artifactId>jackson-databind</artifactId>
32+
</dependency>
33+
34+
<dependency>
35+
<groupId>org.elasticmq</groupId>
36+
<artifactId>elasticmq-rest-sqs_2.11</artifactId>
37+
<version>0.13.9</version>
38+
</dependency>
39+
40+
<dependency>
41+
<groupId>com.jashmore</groupId>
42+
<artifactId>java-dynamic-sqs-listener-core</artifactId>
43+
<version>${project.version}</version>
44+
</dependency>
45+
</dependencies>
46+
</project>

0 commit comments

Comments
 (0)