Skip to content

Commit b8b852d

Browse files
tim-aerotim-aero
authored
Added configuration in code (#147)
* Added configuration in code - Builder pattern used to build configuration - Refactored the AeroMapper and ReactiveAeroMapper to extracts the builders into a common class whilse preserving the same interface - Added setters onto several configuration classes - Added unit tests for the configuration in code - Minor other tweaks * Fixed issues from last PR Refactored the ClassConfig builder to be a separate builder for a cleaner experience * Added code to try to resolve server test errors
1 parent c584e2a commit b8b852d

15 files changed

+773
-382
lines changed

‎README.md

+45
Original file line numberDiff line numberDiff line change
@@ -2023,6 +2023,51 @@ The reference structure is used when the object being referenced is not to be em
20232023
- **batchLoad**: Boolean, defaults to true. When the parent object is loaded, all non-lazy children will also be loaded. If there are several children, it is more efficient to load them from the database using a batch load. if this flag is set to false, children will not be loaded via a batch load. Note that if the parent object has 2 or less children to load, it will single thread the batch load as this is typically more performant than doing a very small batch. Otherwise the batchPolicy on the parent class will dictate how many nodes are hit in the batch at once.
20242024
- **type**: Either ID or DIGEST, defaults to ID. The ID option stores the primary key of the referred object in the referencer, the DIGEST stores the digest instead. Note that DIGEST is not compatible with `lazy=true` as there is nowhere to store the digest. (For example, if the primary key of the object is a long, the digest is 20 bytes, without dynamically creating proxies or subtypes at runtime there is nowhere to store these 20 bytes. Dynamically creating objects like this is not performant so is not allowed).
20252025

2026+
### Configuration through code
2027+
2028+
It is also possible to configure classes through code. This is very useful in situations where external libraries (whose source code is not available) are used and providing all the information in an external configuration file is overkill. This configuration is performed when building the Object Mapper. Let's look at this with an example:
2029+
2030+
```java
2031+
@Data
2032+
@AerospikeRecord(namespace = "test")
2033+
public class A {
2034+
@AerospikeKey
2035+
private long id;
2036+
@AerospikeEmbed(type = AerospikeEmbed.EmbedType.LIST)
2037+
private List<B> b;
2038+
private String aData;
2039+
}
2040+
2041+
@Data
2042+
public class B {
2043+
private C c;
2044+
private String bData;
2045+
}
2046+
2047+
@Data
2048+
public class C {
2049+
private String id;
2050+
private String cData;
2051+
}
2052+
```
2053+
2054+
In this example, let's assume that the source code is available for class `A` but not for either `B` or `C`. If we run this as is, the Object Mapper will not know how to handle the child classes. It will determine that B should be mapped as it's referenced directly from A, but has no idea what to do with C. Using a default builder will throw a `NotSerializableException`.
2055+
2056+
To solve this, we can introduce some configuration in the builder:
2057+
```java
2058+
ClassConfig classConfigC = new ClassConfig.Builder(C.class)
2059+
.withKeyField("id")
2060+
.build();
2061+
ClassConfig classConfigB = new ClassConfig.Builder(B.class)
2062+
.withFieldNamed("c").beingEmbeddedAs(AerospikeEmbed.EmbedType.MAP)
2063+
.build();
2064+
AeroMapper mapper = new AeroMapper.Builder(client)
2065+
.withClassConfigurations(classConfigB, classConfigC)
2066+
.build();
2067+
```
2068+
2069+
In this case we've told the mapper that `B.class` should be treated as an `AerospikeRecord` (`.withConfigurationForClass(B.class)`) and that the 'c' field in that class should be embedded as a MAP. The class `C` is also set to be a mapped class and that the key of that class is to be the field `id`. The class needs to have a key as it's being stored in a map, and objects being stored in a map must be identified by a key.
2070+
20262071
## Virtual Lists
20272072

20282073
When mapping a Java object to Aerospike the most common operations to do are to save the whole object and load the whole object. The AeroMapper is set up primarily for these use cases. However, there are cases where it makes sense to manipulate objects directly in the database, particularly when it comes to manipulating lists and maps. This functionality is provided via virtual lists.

‎src/main/java/com/aerospike/mapper/tools/AbstractBuilder.java

+244
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,244 @@
1+
package com.aerospike.mapper.tools;
2+
3+
import java.io.BufferedReader;
4+
import java.io.File;
5+
import java.io.IOException;
6+
import java.io.InputStream;
7+
import java.io.InputStreamReader;
8+
import java.util.ArrayList;
9+
import java.util.Arrays;
10+
import java.util.List;
11+
import java.util.Set;
12+
import java.util.stream.Collectors;
13+
14+
import javax.validation.constraints.NotNull;
15+
16+
import org.apache.commons.lang3.StringUtils;
17+
18+
import com.aerospike.client.AerospikeException;
19+
import com.aerospike.client.Log;
20+
import com.aerospike.client.policy.BatchPolicy;
21+
import com.aerospike.client.policy.Policy;
22+
import com.aerospike.client.policy.QueryPolicy;
23+
import com.aerospike.client.policy.ScanPolicy;
24+
import com.aerospike.mapper.annotations.AerospikeRecord;
25+
import com.aerospike.mapper.tools.ClassCache.PolicyType;
26+
import com.aerospike.mapper.tools.configuration.ClassConfig;
27+
import com.aerospike.mapper.tools.configuration.Configuration;
28+
import com.aerospike.mapper.tools.utils.TypeUtils;
29+
import com.fasterxml.jackson.core.JsonProcessingException;
30+
import com.fasterxml.jackson.databind.ObjectMapper;
31+
import com.fasterxml.jackson.dataformat.yaml.YAMLFactory;
32+
33+
public abstract class AbstractBuilder<T extends IBaseAeroMapper> {
34+
private final T mapper;
35+
private List<Class<?>> classesToPreload = null;
36+
37+
protected AbstractBuilder(T mapper) {
38+
this.mapper = mapper;
39+
}
40+
/**
41+
* Add in a custom type converter. The converter must have methods which implement the ToAerospike and FromAerospike annotation.
42+
*
43+
* @param converter The custom converter
44+
* @return this object
45+
*/
46+
public AbstractBuilder<T> addConverter(Object converter) {
47+
GenericTypeMapper typeMapper = new GenericTypeMapper(converter);
48+
TypeUtils.addTypeMapper(typeMapper.getMappedClass(), typeMapper);
49+
50+
return this;
51+
}
52+
53+
public AbstractBuilder<T> preLoadClasses(Class<?>... clazzes) {
54+
if (classesToPreload == null) {
55+
classesToPreload = new ArrayList<>();
56+
}
57+
classesToPreload.addAll(Arrays.asList(clazzes));
58+
return this;
59+
}
60+
61+
public String getPackageName(Class<?> clazz) {
62+
Class<?> c;
63+
if (clazz.isArray()) {
64+
c = clazz.getComponentType();
65+
} else {
66+
c = clazz;
67+
}
68+
String pn;
69+
if (c.isPrimitive()) {
70+
pn = "java.lang";
71+
} else {
72+
String cn = c.getName();
73+
int dot = cn.lastIndexOf('.');
74+
pn = (dot != -1) ? cn.substring(0, dot).intern() : "";
75+
}
76+
return pn;
77+
}
78+
79+
public AbstractBuilder<T> preLoadClassesFromPackage(Class<?> classInPackage) {
80+
return preLoadClassesFromPackage(getPackageName(classInPackage));
81+
}
82+
83+
public AbstractBuilder<T> preLoadClassesFromPackage(String thePackage) {
84+
Set<Class<?>> clazzes = findAllClassesUsingClassLoader(thePackage);
85+
for (Class<?> thisClazz : clazzes) {
86+
// Only add classes with the AerospikeRecord annotation.
87+
if (thisClazz.getAnnotation(AerospikeRecord.class) != null) {
88+
this.preLoadClass(thisClazz);
89+
}
90+
}
91+
return this;
92+
}
93+
94+
// See https://www.baeldung.com/java-find-all-classes-in-package
95+
private Set<Class<?>> findAllClassesUsingClassLoader(String packageName) {
96+
InputStream stream = ClassLoader.getSystemClassLoader()
97+
.getResourceAsStream(packageName.replaceAll("[.]", "/"));
98+
BufferedReader reader = new BufferedReader(new InputStreamReader(stream));
99+
return reader.lines().filter(line -> line.endsWith(".class")).map(line -> getClass(line, packageName))
100+
.collect(Collectors.toSet());
101+
}
102+
103+
private Class<?> getClass(String className, String packageName) {
104+
try {
105+
return Class.forName(packageName + "." + className.substring(0, className.lastIndexOf('.')));
106+
} catch (ClassNotFoundException ignored) {
107+
}
108+
return null;
109+
}
110+
111+
public AbstractBuilder<T> preLoadClass(Class<?> clazz) {
112+
if (classesToPreload == null) {
113+
classesToPreload = new ArrayList<>();
114+
}
115+
classesToPreload.add(clazz);
116+
return this;
117+
}
118+
119+
public AbstractBuilder<T> withConfigurationFile(File file) throws IOException {
120+
return this.withConfigurationFile(file, false);
121+
}
122+
123+
public AbstractBuilder<T> withConfigurationFile(File file, boolean allowsInvalid) throws IOException {
124+
ObjectMapper objectMapper = new ObjectMapper(new YAMLFactory());
125+
Configuration configuration = objectMapper.readValue(file, Configuration.class);
126+
this.loadConfiguration(configuration, allowsInvalid);
127+
return this;
128+
}
129+
130+
public AbstractBuilder<T> withConfigurationFile(InputStream ios) throws IOException {
131+
return this.withConfigurationFile(ios, false);
132+
}
133+
134+
public AbstractBuilder<T> withConfigurationFile(InputStream ios, boolean allowsInvalid) throws IOException {
135+
ObjectMapper objectMapper = new ObjectMapper(new YAMLFactory());
136+
Configuration configuration = objectMapper.readValue(ios, Configuration.class);
137+
this.loadConfiguration(configuration, allowsInvalid);
138+
return this;
139+
}
140+
141+
public AbstractBuilder<T> withConfiguration(String configurationYaml) throws JsonProcessingException {
142+
return this.withConfiguration(configurationYaml, false);
143+
}
144+
145+
public AbstractBuilder<T> withConfiguration(String configurationYaml, boolean allowsInvalid) throws JsonProcessingException {
146+
ObjectMapper objectMapper = new ObjectMapper(new YAMLFactory());
147+
Configuration configuration = objectMapper.readValue(configurationYaml, Configuration.class);
148+
this.loadConfiguration(configuration, allowsInvalid);
149+
return this;
150+
}
151+
152+
public AbstractBuilder<T> withClassConfigurations(ClassConfig classConfig, ClassConfig ...classConfigs) {
153+
Configuration configuration = new Configuration();
154+
configuration.add(classConfig);
155+
for (ClassConfig thisConfig : classConfigs) {
156+
configuration.add(thisConfig);
157+
}
158+
ClassCache.getInstance().addConfiguration(configuration);
159+
return this;
160+
161+
}
162+
163+
private void loadConfiguration(@NotNull Configuration configuration, boolean allowsInvalid) {
164+
for (ClassConfig config : configuration.getClasses()) {
165+
try {
166+
String name = config.getClassName();
167+
if (StringUtils.isBlank(name)) {
168+
throw new AerospikeException("Class with blank name in configuration file");
169+
} else {
170+
try {
171+
Class.forName(config.getClassName());
172+
} catch (ClassNotFoundException e) {
173+
throw new AerospikeException("Cannot find a class with name " + name);
174+
}
175+
}
176+
} catch (RuntimeException re) {
177+
if (allowsInvalid) {
178+
Log.warn("Ignoring issue with configuration: " + re.getMessage());
179+
} else {
180+
throw re;
181+
}
182+
}
183+
}
184+
ClassCache.getInstance().addConfiguration(configuration);
185+
}
186+
187+
public static class AeroPolicyMapper<T extends IBaseAeroMapper> {
188+
private final AbstractBuilder<T> builder;
189+
private final Policy policy;
190+
private final PolicyType policyType;
191+
192+
public AeroPolicyMapper(AbstractBuilder<T> builder, PolicyType policyType, Policy policy) {
193+
this.builder = builder;
194+
this.policyType = policyType;
195+
this.policy = policy;
196+
}
197+
198+
public AbstractBuilder<T> forClasses(Class<?>... classes) {
199+
for (Class<?> thisClass : classes) {
200+
ClassCache.getInstance().setSpecificPolicy(policyType, thisClass, policy);
201+
}
202+
return builder;
203+
}
204+
205+
public AbstractBuilder<T> forThisOrChildrenOf(Class<?> clazz) {
206+
ClassCache.getInstance().setChildrenPolicy(this.policyType, clazz, this.policy);
207+
return builder;
208+
}
209+
210+
public AbstractBuilder<T> forAll() {
211+
ClassCache.getInstance().setDefaultPolicy(policyType, policy);
212+
return builder;
213+
}
214+
}
215+
216+
public AeroPolicyMapper<T> withReadPolicy(Policy policy) {
217+
return new AeroPolicyMapper<>(this, PolicyType.READ, policy);
218+
}
219+
220+
public AeroPolicyMapper<T> withWritePolicy(Policy policy) {
221+
return new AeroPolicyMapper<>(this, PolicyType.WRITE, policy);
222+
}
223+
224+
public AeroPolicyMapper<T> withBatchPolicy(BatchPolicy policy) {
225+
return new AeroPolicyMapper<>(this, PolicyType.BATCH, policy);
226+
}
227+
228+
public AeroPolicyMapper<T> withScanPolicy(ScanPolicy policy) {
229+
return new AeroPolicyMapper<>(this, PolicyType.SCAN, policy);
230+
}
231+
232+
public AeroPolicyMapper<T> withQueryPolicy(QueryPolicy policy) {
233+
return new AeroPolicyMapper<>(this, PolicyType.QUERY, policy);
234+
}
235+
236+
public T build() {
237+
if (classesToPreload != null) {
238+
for (Class<?> clazz : classesToPreload) {
239+
ClassCache.getInstance().loadClass(clazz, this.mapper);
240+
}
241+
}
242+
return this.mapper;
243+
}
244+
}

0 commit comments

Comments
 (0)