Skip to content

Commit 3f3b169

Browse files
committed
Bump stable to 2.24.0
1 parent ab5d283 commit 3f3b169

File tree

7 files changed

+112
-83
lines changed

7 files changed

+112
-83
lines changed

README.md

+91-62
Original file line numberDiff line numberDiff line change
@@ -25,59 +25,88 @@ FormShare **is for organizations** to install it in their server or cloud servic
2525

2626
## Features
2727

28-
**Current features**
29-
30-
- Case management (Longitudinal data collection) (Version >= 2.8.0)
31-
- Using the Official ODK Collect App
32-
- Intelligent work flow using Official ODK standards:
33-
- Case creator forms will create cases.
34-
- Follow-up forms will attach information to each case.
35-
- Deactivate forms will deactivate cases. For example, a household that decides to exit a longitudinal study will not appear in follow-up forms after deactivation.
36-
- Activate forms will activate cases again. For example, a household that decides to re-enter a longitudinal study will appear again in follow-up forms after activation.
37-
- Move information from case creator forms into follow-up, deactivation and activation forms. For example, the sex of a participant (e. g., female) could be used in follow-up case forms to ask specific questions according to sex (e. g., if female, do they have access to reproductive health services since our last visit?)
38-
- Support multiple time zones:
39-
- FormShare's: This is the time zone of the Linux server running FormShare.
40-
- User: This is the time zone of a user logged into FormShare.
41-
- Project: Each project can have a different time zone. This should be the time zone where submissions happen.
42-
- Assistant: This is the time zone of an assistant logged into FormShare.
43-
- Partner: This is the time zone of a partner logged into FormShare.
44-
45-
- User accounts and management
46-
- Group-based user permissions
47-
- Projects to organize users, permissions, and forms
48-
- Separate access for data collectors and data cleaners. This is useful when dealing with hundreds of data collectors that do not need a FormShare user access
49-
- Separate access for partners: Partners are trusted individuals outside your organization that require access to products and other resources in FormShare to collaborate with you. For example, a professor at a university (a partner in your project) requiring a KML export to match GPS points with weather data.
50-
- User collaborations at project level, e.g., you can allow a colleague to maintain certain aspects of your project.
51-
- Form and submission management
52-
- With easy setup of ODK Collect Projects using QR images
53-
- With support for form version updates
54-
- With testing and production stages
55-
- Showing structural changes between incremental versions.
56-
- With form and submission multimedia or data attachments
57-
- With a table preview of submission data (even with thousand or millions of records) allowing in-table edits and recording any changes made to the data
58-
- OData live data feed for analysis with tools like Excel and Power BI. **With all CRUD operation supported (e. g., update)**. You can even use [Excel](https://github.com/qlands/MrBot-OData-Add-In) to clean data.
59-
- Extensibility system, e.g., You can write extensions to connect FormShare with Microsoft 365 authentication system
60-
- Documentation for running on AWS using Docker
61-
- Data cleaning API integration with R, STATA, or SPSS
62-
- Real-time map visualization of geo-referenced submissions at project and form level
63-
- Fill out forms using the web browser through [Enketo](https://github.com/qlands/formshare_enketo_plugin)
64-
- Download form attachments
65-
- Filtering submissions by submission metadata (e.g., date and time received on server)
66-
- Parse submissions into database tables. FormShare will create a MySQL data repository to store your data
67-
- The repository is controlled with a primary key e.g., Farmed ID
68-
- Duplicated submissions go to a cleaning pipe-line system that makes it easier to compare submissions and decide what to do with the duplicates
69-
- Private and publishable data downloads in Excel and CSV formats
70-
- Mark data fields as sensitive to exclude them from publishable products
71-
- Version control on products, e.g., FormShare will let you know if a data export does not have the latest submissions or changes in the database
72-
- Make publishable products publicly available knowing that sensitive fields are excluded automatically
73-
- Download geo-referenced information in KML format
74-
75-
**Short-term features:**
76-
77-
- Graph visualization with dashboards.
78-
- Connecting data fields with ontological variables. This is useful when comparing variables across studies even if variable names are different
79-
- Real-time data aggregation (pull data from different forms into one common data bucket). This is useful when dealing with slightly different forms for different geographies but where that certain fields could be aggregated into a common pot for analysis
80-
- Real-time data cleaning scripts using R
28+
### Case management (Longitudinal data collection) (Version >= 2.8.0)
29+
30+
- Using the Official ODK Collect App.
31+
- Intelligent workflow using Official ODK standards:
32+
- Case creator forms will create cases.
33+
- Follow-up forms will attach information to each case.
34+
- Deactivate forms will deactivate cases. For example, a household that decides to exit a longitudinal study will not appear in follow-up forms after deactivation.
35+
- Activate forms will activate cases again. For example, a household that decides to re-enter a longitudinal study will appear again in follow-up forms after activation.
36+
- Move information from case creator forms into follow-up, deactivation, and activation forms. For example, the sex of a participant (e. g., female) could be used in follow-up case forms to ask specific questions according to sex (e. g., if female, do they have access to reproductive health services since our last visit?)
37+
38+
### Support for multiple time zones
39+
40+
FormShare can produce dates and times in the following time zones:
41+
42+
- FormShare: This is the time zone of the Linux server running FormShare.
43+
- User: This is the time zone of a user logged into FormShare.
44+
- Project: Each project can have a different time zone. This should be the time zone where submissions happen.
45+
- Assistant: This is the time zone of an assistant logged into FormShare.
46+
- Partner: This is the time zone of a partner logged into FormShare.
47+
48+
### User and project management
49+
50+
- User accounts and management.
51+
- Group-based user permissions.
52+
- Projects to organize users, permissions, and forms.
53+
- Separate access for data collectors and data cleaners. This is useful when dealing with hundreds of data collectors that do not need a FormShare user's account.
54+
- Separate access for partners: Partners are trusted individuals outside your organization that require access to products and other resources in FormShare to collaborate with you. For example, a professor at a university (a partner in your project) requires a KML export to match GPS points with weather data.
55+
- User collaborations at a project level, e.g., you can allow a colleague to maintain certain aspects of your project.
56+
57+
### Form and submission management
58+
59+
- Easy setup of ODK Collect Projects using QR images.
60+
- Support for form version updates.
61+
- With testing and production stages.
62+
- Showing structural changes between incremental versions.
63+
- With form and submission multimedia or data attachments.
64+
- Table preview of submission data (even with thousands or millions of records) allowing in-table edits and recording any changes made to the data.
65+
- Fill out forms using the web browser through [Enketo](https://github.com/qlands/formshare_enketo_plugin).
66+
- Real-time map visualization of geo-referenced submissions at project and form levels.
67+
- Download form attachments.
68+
69+
### Database management and interoperability
70+
71+
- Parse submissions into database tables. FormShare will create a MySQL data repository to store your data.
72+
- The repository is controlled with a primary key e.g., Farmed ID.
73+
- Duplicated submissions go to a cleaning pipeline system that makes it easier to compare submissions and decide what to do with the duplicates.
74+
- Filtering submissions by submission metadata (e.g., date and time received on server).
75+
- Data cleaning API integration with R, STATA, or SPSS.
76+
- OData real-time data feed for analysis with tools like Excel and Power BI. **With all CRUD operations supported (e. g., update)**. You can even use [Excel](https://github.com/qlands/MrBot-OData-Add-In) to clean data.
77+
- Data dictionary with personal information protection.
78+
79+
### Product management
80+
81+
- Private and publishable data downloads in Excel, CSV, and JSON formats.
82+
- Publishable products exclude protected fields (personal information protection) automatically.
83+
- Version control on products, e.g., FormShare will let you know if a data export does not have the latest submissions or changes in the database.
84+
- Download geo-referenced information in KML format.
85+
86+
### Performance, concurrency, and parallelism
87+
88+
Though FormShare with the default settings can handle a load that would fit most organizations, it can be configured to handle any load. FormShare has been tested, using [JMeter](https://jmeter.apache.org/), under extreme traffic (1000 parallel submissions ) in an eight years old laptop with 16 GB of RAM and an [i7-4712HQ](https://www.intel.com/content/www/us/en/products/sku/78932/intel-core-i74712hq-processor-6m-cache-up-to-3-30-ghz/specifications.html).
89+
90+
- Each submission is transactional. This means that a submission is either processed completely or thoroughly discarded for ODK Collect to re-send it.
91+
- Under extreme traffic, FormShare can store a complex survey like [RHoMIS](https://www.rhomis.org/) at 18 submissions per second with zero failures.
92+
- Data exports support concurrent processing. A survey like RHoMIS with 100,000 submissions and millions of rows would take less than a minute to export to JSON or CSV.
93+
- The performance of the user interface or the data cleaning interface is not affected by the number of submissions.
94+
95+
### Analytics
96+
97+
- Real-time analytics from R using the [FormShare R Package](https://cran.r-project.org/web/packages/FormShare/index.html).
98+
- Real-time analytics from platforms like [Tableau](https://public.tableau.com/en-us/s/) and [Power BI](https://powerbi.microsoft.com/en-us/) using the [FormShare OData extension](https://github.com/qlands/formshare_odata_plugin).
99+
100+
### Extensibility and others
101+
102+
- Extensibility system, e.g., You can write extensions to connect FormShare with Microsoft 365 authentication system. See a list of extensions [below](#Customization-and-Extension).
103+
- Documentation for running on AWS using Docker.
104+
105+
### Future features
106+
107+
- Simple graph visualization with dashboards.
108+
- Connecting data fields with ontological variables. This is useful when comparing variables across studies even if variable names are different.
109+
- Real-time data aggregation (pull data from different forms into one common data bucket). This is useful when dealing with slightly different forms for different geographies but where that certain fields could be aggregated into a common pot for analysis.
81110

82111
ScreenShot
83112
----------
@@ -86,11 +115,11 @@ ScreenShot
86115

87116
Releases
88117
------------
89-
The current stable release is 2.23.0 and it is available [here](https://github.com/qlands/FormShare/tree/stable-2.23.0)
118+
The current stable release is 2.24.0 and it is available [here](https://github.com/qlands/FormShare/tree/stable-2.24.0)
90119

91-
The database signature for stable 2.23.0 is e2d6d5aa3700
120+
The database signature for stable 2.24.0 is e2d6d5aa3700
92121

93-
The Docker image for stable 2.23.0 is 20221219
122+
The Docker image for stable 2.24.0 is 20221229
94123

95124
Installation
96125
------------
@@ -110,9 +139,9 @@ sudo apt-get install -y docker-compose
110139

111140
# Get the Docker Compose file
112141
cd /opt
113-
sudo mkdir formshare_docker_compose_20221219
114-
cd formshare_docker_compose_20221219
115-
sudo wget https://raw.githubusercontent.com/qlands/FormShare/stable-2.23.0/docker_compose/docker-compose.yml
142+
sudo mkdir formshare_docker_compose_20221229
143+
cd formshare_docker_compose_20221229
144+
sudo wget https://raw.githubusercontent.com/qlands/FormShare/stable-2.24.0/docker_compose/docker-compose.yml
116145

117146
# Make the directory structure for FormShare
118147
sudo mkdir /opt/formshare
@@ -137,11 +166,11 @@ sudo sysctl -w vm.max_map_count=262144
137166
echo 'vm.max_map_count=262144' | sudo tee -a /etc/sysctl.d/60-vm-max_map_count.conf
138167

139168
# Download all the required Docker Images
140-
cd /opt/formshare_docker_compose_20221219
169+
cd /opt/formshare_docker_compose_20221229
141170
sudo docker-compose pull
142171

143172
# Edit the docker-compose.yml file to set the MySQL root and FormShare admin passwords
144-
sudo nano /opt/formshare_docker_compose_20221219/docker-compose.yml
173+
sudo nano /opt/formshare_docker_compose_20221229/docker-compose.yml
145174
# Press Alt+Shit+3 to show the line numbers in Nano
146175

147176
Edit line 10: Change the root password from "my_secure_password" to your password
@@ -189,7 +218,7 @@ sudo service apache2 start
189218
# Subsequent start will take about 2 minutes. You can check the status with "sudo docker stats".
190219
# FormShare will be ready for usage when the container reaches more than 500 kB of MEM USAGE
191220
# This is the only two commands you need to start FormShare after a server restart
192-
cd /opt/formshare_docker_compose_20221219
221+
cd /opt/formshare_docker_compose_20221229
193222
sudo docker-compose up -d
194223

195224
# Browse to FormShare

docker_compose/docker-compose.yml

+15-15
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,9 @@
11
version: '3'
22
services:
3-
fsmysql_20221219:
3+
fsmysql_20221229:
44
image: mysql:8.0.23
55
command: --default-authentication-plugin=mysql_native_password
6-
container_name: fs_mysql_20221219
6+
container_name: fs_mysql_20221229
77
cap_add:
88
- SYS_NICE
99
environment:
@@ -14,15 +14,15 @@ services:
1414
fsnet:
1515
ipv4_address: 172.28.1.5
1616

17-
fses20221219n01:
17+
fses20221229n01:
1818
image: docker.elastic.co/elasticsearch/elasticsearch:7.14.2
19-
container_name: fses20221219n01
19+
container_name: fses20221229n01
2020
environment:
2121
- network.host=172.28.1.1
22-
- node.name=fses20221219n01
22+
- node.name=fses20221229n01
2323
- cluster.name=fs-es-cluster
24-
- discovery.seed_hosts=fses20221219n02
25-
- cluster.initial_master_nodes=fses20221219n01,fses20221219n02
24+
- discovery.seed_hosts=fses20221229n02
25+
- cluster.initial_master_nodes=fses20221229n01,fses20221229n02
2626
- bootstrap.memory_lock=true
2727
- xpack.security.enabled=false
2828
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
@@ -36,15 +36,15 @@ services:
3636
fsnet:
3737
ipv4_address: 172.28.1.1
3838

39-
fses20221219n02:
39+
fses20221229n02:
4040
image: docker.elastic.co/elasticsearch/elasticsearch:7.14.2
41-
container_name: fses20221219n02
41+
container_name: fses20221229n02
4242
environment:
4343
- network.host=172.28.1.2
44-
- node.name=fses20221219n02
44+
- node.name=fses20221229n02
4545
- cluster.name=fs-es-cluster
46-
- discovery.seed_hosts=fses20221219n01
47-
- cluster.initial_master_nodes=fses20221219n01,fses20221219n02
46+
- discovery.seed_hosts=fses20221229n01
47+
- cluster.initial_master_nodes=fses20221229n01,fses20221229n02
4848
- bootstrap.memory_lock=true
4949
- xpack.security.enabled=false
5050
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
@@ -58,9 +58,9 @@ services:
5858
fsnet:
5959
ipv4_address: 172.28.1.2
6060

61-
formshare_20221219:
62-
image: qlands/formshare2:20221219
63-
container_name: formshare_20221219
61+
formshare_20221229:
62+
image: qlands/formshare2:20221229
63+
container_name: formshare_20221229
6464
environment:
6565
MYSQL_HOST_NAME: 172.28.1.5
6666
MYSQL_USER_NAME: root
+1-1
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
sudo docker build --no-cache --rm -t qlands/formshare2:20221219 -f ./stable.Dockerfile .
1+
sudo docker build --no-cache --rm -t qlands/formshare2:20221229 -f ./stable.Dockerfile .

docker_files/formshare/stable.Dockerfile

+2-2
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
FROM qlands/odktools:20221218
1+
FROM qlands/odktools:20221228
22

33
MAINTAINER QLands Technology Consultants
44

@@ -30,7 +30,7 @@ COPY ./docker_files/mosquitto/access.acl /root
3030
RUN mkdir formshare_gunicorn
3131
RUN python3 -m venv formshare_env
3232

33-
RUN git clone https://github.com/qlands/FormShare.git -b stable-2.23.0 formshare
33+
RUN git clone https://github.com/qlands/FormShare.git -b stable-2.24.0 formshare
3434
RUN . ./formshare_env/bin/activate && pip install wheel && pip install -r /opt/formshare/requirements.txt && python /opt/formshare/download_nltk_packages.py
3535

3636
ADD https://github.com/ufoscout/docker-compose-wait/releases/download/2.6.0/wait /wait

formshare/plugins/helpers.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -113,7 +113,7 @@ def get_version():
113113
This returns the version of FormShare
114114
:return: The version of FormShare
115115
"""
116-
return "2.23.0 (20221219)"
116+
return "2.24.0 (20221229)"
117117

118118

119119
@core_helper

install_steps.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -257,7 +257,7 @@ sudo chown -R $whoami formshare_env
257257

258258
```sh
259259
cd /opt
260-
sudo git clone https://github.com/qlands/FormShare.git -b stable-2.23.0 formshare
260+
sudo git clone https://github.com/qlands/FormShare.git -b stable-2.24.0 formshare
261261
sudo chown -R $whoami formshare
262262
```
263263

setup.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -173,7 +173,7 @@
173173

174174
setup(
175175
name="formshare",
176-
version="2.23.0",
176+
version="2.24.0",
177177
description="FormShare",
178178
long_description=README + "\n\n" + CHANGES,
179179
classifiers=[

0 commit comments

Comments
 (0)