Skip to content

New jamtools main #134

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 129 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
129 commits
Select commit Hold shift + click to select a range
0038770
pre diving deep to fine connection error
Jul 2, 2024
2bdd0cf
mid fix, semi working
Jul 3, 2024
e8e3bb2
first draft
Jul 3, 2024
58030b5
first working draft for device/cloud/fog level
Jul 3, 2024
9bcdead
bg working
Jul 3, 2024
077675d
pre varName Refactor
Jul 4, 2024
82f6a14
pre file directory abstraction
Jul 5, 2024
6387306
checkpoint
Jul 5, 2024
f734ea6
pre localizing variables
Jul 5, 2024
26c656f
Broken after jappid changes
Jul 5, 2024
2a61745
pre meeting
Jul 5, 2024
d233e35
blocking bg jamrun fixed
Jul 9, 2024
cb0f28d
cleanUp problem fixed
Jul 10, 2024
7f34694
resume arg added
Jul 10, 2024
b1324de
num issue is fixed,tmux running in the right directory, file creation…
Jul 11, 2024
be9609b
Fog and Cloud breaking fixed
Jul 11, 2024
e8cd997
list kill working first itter
Jul 15, 2024
e4f04f6
jamClean big issues fixed first running iter
Jul 15, 2024
a6d2fc4
start of the file by file finilizing
Jul 16, 2024
24a1356
checkpoint
Jul 16, 2024
bac632b
checkPoint
Jul 16, 2024
94747b9
port issue is resolved for the device
Jul 16, 2024
af8ce5b
mqtt redundent parent process fixed
Jul 17, 2024
a26dafb
tmux kill fixed
Jul 17, 2024
4800b16
checkpoint broken group
Jul 18, 2024
9603b8d
Pre fixing mosquitto
Jul 19, 2024
78cdb95
jamKill refactor
Jul 22, 2024
bbc63b3
jamkill refactored
Jul 23, 2024
eae2b24
monitoring added, refactored jamlist
Jul 24, 2024
cc17ab0
jamClean added
Jul 25, 2024
d2048f0
cleaning included/ monitoring improved
Jul 25, 2024
0afeaa9
interval added
Jul 25, 2024
1369096
live demo checkpoint
Jul 25, 2024
55b6f35
quick fixes
Jul 25, 2024
6fd49ad
before pause(potential revert required)
Jul 26, 2024
0fc3c65
pre resume changes
Jul 26, 2024
4a58733
remote changes stashed
Jul 26, 2024
bb8aba6
added the usage(not final v)
Jul 27, 2024
e501249
relative path issue fixded, dockerFile Updated
Jul 29, 2024
48db3aa
mainCompiler UptoDate
Jul 29, 2024
2a244db
install zx globaly
Jul 30, 2024
9b106a3
fixed file errors, docker working
Jul 30, 2024
fd25d83
aiming to fix the ssh not finding the shell issue
Jul 30, 2024
da8e761
attemp to expose the ports for network connections
Jul 30, 2024
e859e5f
remove port expose
Jul 30, 2024
b82160c
ulternative dockerfile, redis blocker
Jul 31, 2024
9cbadca
docker fixed
Jul 31, 2024
a509f14
working jamlist/jamclean/jamkill/jamrun for remote machin(DOCKER VERS…
Aug 2, 2024
7e72a88
remoteLog
Aug 3, 2024
58f811d
usage update
Aug 3, 2024
4af06ca
minor fixes pre undockerizing the config
Aug 5, 2024
934a543
resume fixed local only
Aug 5, 2024
eb8dbfc
remote resume fixed
Aug 5, 2024
245fed1
quick fix
Aug 6, 2024
785170e
jam run resume path fix
Aug 6, 2024
29688a2
PATCH FIX, USAGE UPDATE, CLEANUP
Aug 7, 2024
86653c3
USage fix
Aug 8, 2024
c4457ac
cleanUp for file directory, removing .mjs extention
Aug 8, 2024
cf6ad1f
executable scripts
Aug 8, 2024
c715db1
shabang fixed, error not throw but just logged
Aug 8, 2024
b256530
console.log cleanup
Aug 8, 2024
037df70
remove set of extra logs
Aug 8, 2024
39aa879
corrupted portDir fic
Aug 9, 2024
498ae1c
no ext -> mjs
Aug 9, 2024
8eb85fe
jamkill filtering added, all usage updated but jamlist
Aug 9, 2024
972a278
jamlist and jamkill args to '==' format and --app, --prog, --port
Aug 9, 2024
71df879
jamlist monitoring fixed, jamlist usage updated
Aug 9, 2024
5921a87
format fix, jamlist fix, uptime added
Aug 10, 2024
35f39c2
patch fix -- old version of jamterm cherry picked
Aug 10, 2024
ae8431b
modified the install scripts
anrl Aug 11, 2024
a605ff5
removed the resume and pause
anrl Aug 12, 2024
1d0c1f7
potential ENOENT FIX in jamlist
Aug 12, 2024
c563f62
jamkill new arg format fixed in jamclean
Aug 12, 2024
880b531
applied the patch from shahin
anrl Aug 13, 2024
cff7e95
jamterm itter1
Aug 14, 2024
a5dc3b3
Merge branch 'feat-new-jamtools' of https://github.com/citelab/JAMScr…
Aug 14, 2024
82ebb36
tmux split error fixed
Aug 15, 2024
6f8a2d4
old jamterm deleted
anrl Aug 15, 2024
d651efe
filter fixed
Aug 16, 2024
d9a8be1
Merge branch 'feat-new-jamtools' of https://github.com/citelab/JAMScr…
Aug 16, 2024
8abd0b6
prog filter fix
Aug 16, 2024
3074848
jamterm and jamBatch completed
Aug 27, 2024
b8a0e31
config jambatch added
Aug 29, 2024
e8746c1
logging improved. minor bug fix
Aug 30, 2024
ee64601
made jambatch.mjs exec
anrl Aug 30, 2024
79504d1
fixed jambatch
Aug 30, 2024
2a399ba
jambatch fix 2
Aug 30, 2024
237a513
CleanUp
Aug 30, 2024
ec4d62d
tmux fix and added on fly modification for workers
Sep 13, 2024
edc1994
Modified app.js to include api calls
JeanKa25 Nov 6, 2024
c3059a7
Done jam batch/log/list
andrei-sandor Nov 8, 2024
4a6260f
Done jam kill
JasonBergeron Nov 9, 2024
cc0874a
fix from get to post
andrei-sandor Nov 10, 2024
c4ee252
jamterm post API request and tmux running detached.
jininij Nov 13, 2024
cface11
All endpoints work!
JeanKa25 Nov 14, 2024
c65705e
Fixed Dockerfile!
JeanKa25 Nov 24, 2024
101f9f8
Added executables for jt1.jxe and jt2.jxe
JeanKa25 Nov 26, 2024
becc418
Added executables for jt1.jxe and jt2.jxe
JeanKa25 Nov 26, 2024
fd0154f
start of wrapper
andrei-sandor Dec 24, 2024
61cc706
finish wrapper
andrei-sandor Dec 24, 2024
fc25e2d
clean
andrei-sandor Dec 27, 2024
7c8f053
added for exec
andrei-sandor Dec 28, 2024
b50f7ba
Update wrapper.mjs
andrei-sandor Dec 28, 2024
b40a8e6
Finish the wrapper
andrei-sandor Jan 4, 2025
2fd370d
fix index issue
andrei-sandor Jan 4, 2025
04a59d1
jambatch works
andrei-sandor Jan 5, 2025
b7922a3
all endpoints do work
andrei-sandor Jan 5, 2025
58f45ea
fix == issues
andrei-sandor Jan 5, 2025
9eddcd5
Added docker files
JeanKa25 Jan 12, 2025
d79da98
Fixed JAMRun
JeanKa25 Jan 14, 2025
dfd18bd
docker works
andrei-sandor Jan 16, 2025
1d9a4a0
Added app-docker
JeanKa25 Jan 16, 2025
af97b29
done app-docker
andrei-sandor Jan 16, 2025
fd1f17e
Finish djam server
andrei-sandor Jan 19, 2025
d7ab126
ansible setup script
JeanKa25 Jan 20, 2025
5eab45f
Update app.js path
JeanKa25 Jan 22, 2025
0c91873
Fixed Ansible
JeanKa25 Jan 22, 2025
9123a58
djamrun djamterm(partially done) and playbook commits
jininij Mar 2, 2025
92d1163
setting relative path to absolute path for ansible adhoc command for …
jininij Mar 6, 2025
949e390
updated djam run, term, and list for fog and cloud
jininij Mar 13, 2025
fb529b8
Remove redundant ID column duplicated by NAME in djamlist output
jininij Mar 13, 2025
a93cd08
ssh for app.js
jininij Mar 13, 2025
ea4975b
ssh for app.js
jininij Mar 13, 2025
66e4d38
wrapper update to have serverIP input
jininij Mar 26, 2025
df50a62
Add performance test scripts (jamrun and jamlist) and resource monito…
jininij Mar 27, 2025
29e0028
updating ansible playbooks and adding performance test scripts
jininij Apr 7, 2025
ae7ffb2
Update README.md
JeanKa25 Apr 9, 2025
ca24435
Update README.md
JeanKa25 Apr 9, 2025
2bd1807
Update README.md
JeanKa25 Apr 10, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 87 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
FROM ubuntu

# Install SSH and other dependencies
RUN apt update && apt install -y openssh-server
RUN sed -i 's/PermitRootLogin prohibit-password/PermitRootLogin yes/' /etc/ssh/sshd_config

# Install necessary tools and libraries
RUN apt-get update && apt-get install -y --no-install-recommends apt-utils && \
apt-get install -y -q wget \
build-essential \
python3 \
unzip \
libbsd-dev \
git \
sudo \
vim \
curl \
make \
clang \
iproute2 \
net-tools \
unzip \
iputils-ping \
mosquitto \
mosquitto-clients \
libmosquitto-dev \
redis-server \
redis-tools \
libhiredis-dev \
libevent-dev \
tmux \
inotify-tools

# Create an admin user with sudo privileges
RUN useradd -m admin && echo "admin:admin" | chpasswd && adduser admin sudo
RUN echo "admin ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers
USER admin

# Set up Node.js
WORKDIR /home/admin
RUN wget https://deb.nodesource.com/setup_20.x && chmod +x ./setup_20.x && echo admin | sudo -S ./setup_20.x && \
sudo apt-get install -y -q nodejs

# Clone JAMScript repository and build dependencies
RUN git clone --branch shahin/migration https://github.com/citelab/JAMScript

WORKDIR /home/admin/JAMScript/deps
RUN git clone https://github.com/intel/tinycbor && \
cd tinycbor && \
make && sudo make install

WORKDIR /home/admin/JAMScript/deps/mujs2
RUN make && sudo make install

# Install JAMScript dependencies and global npm package
WORKDIR /home/admin/JAMScript
RUN npm install
RUN sudo npm install zx -g

# Additional setup for JAMScript
WORKDIR /home/admin/JAMScript/lib/cside
RUN make archive

WORKDIR /home/admin
RUN mkdir -p .jamruns && \
ln -s /home/admin/JAMScript/lib/cside .jamruns/clib && \
ln -s /home/admin/JAMScript .jamruns/jamhome && \
ln -s /home/admin/JAMScript/lib/jside .jamruns/node_modules

# Configure npm global installation path
RUN mkdir .npm-global && mkdir .npm-global/bin
ENV PATH="$PATH:/home/admin/JAMScript/tools:/home/admin/.npm-global/bin"

# Expose SSH port
EXPOSE 22

# Copy the code from the current directory to /home/admin/code in the container
WORKDIR /home/admin/code
COPY . /home/admin/code

# Run npm install and install.sh as admin
RUN npm install
RUN sudo chmod +x /home/admin/code/install.sh && /home/admin/code/install.sh

# Start SSH and keep the container running
ENTRYPOINT sudo service ssh start && tail -f /dev/null

127 changes: 123 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,127 @@
[![Build Status](https://travis-ci.org/citelab/JAMScript.svg?branch=master)](https://travis-ci.org/citelab/JAMScript)
# Redesigning JAMScript Tools for High-Performance and Scalability

This repository is part of an ongoing effort to redesign the JAMScript tools with a focus on high-performance execution and scalable infrastructure.
To achieve this, we've restructured the architecture around a client-server model that separates the responsibilities of coordination and computation.

# A Programming Language and Middleware for Cloud of Moving Things
## 🔧 Getting Started

This project is currently under development. You can see the documentation and install instructions at https://citelab.github.io/JAMScript.
### 1. Clone the Repository

Start by cloning the repository and switching to the appropriate branch:

```bash
git clone https://github.com/JeanKa25/JAMScript.git
```

Then run the install.sh to install all the dependencies:


### 2. Set Environment Variables
Export the following global variables:

```bash
export JAMHOME=~/JAMScript
export PATH=$JAM_HOME/tools:$PATH
export JAMDATA=~/JAMScript/data
```

To make these changes permanent, add them to your .bashrc:

```bash
echo 'export JAMHOME=~/JAMScript' >> ~/.bashrc
echo 'export PATH=$JAM_HOME/tools:$PATH' >> ~/.bashrc
echo 'export JAMDATA=~/JAMScript/data' >> ~/.bashrc
source ~/.bashrc
```


## ⚙️ Installing Dependencies with Ansible

You can set up your environment using Ansible with the provided playbook:

### 1. Install Ansible (if not already installed)

```bash
sudo apt update
sudo apt install ansible -y
```

### 2. Run the setup playbook

```bash
ansible-playbook setup_jamscript.yml -i localhost,
```

> This playbook installs required dependencies, clones the repo, and configures environment variables as needed for JAMScript tools.

---

## 🌐 Running the Server

To start the JAMScript server, navigate to the tools directory and run the server file:

```bash
cd ~/JAMScript/tools
node app-docker.js
```

### 🔧 Path and IP Configuration

Before running the server, make sure to update the following files:

#### `app-docker.js`
- Set the `serverip` parameter to the IP address of the server.

#### `Wrapper.mjs`
- Update the following:
- `serverip`: match the IP set in `app-docker.js`.
- Path to JAMScript tools directory.
- Path to your SSH `authorized_keys` file (used for key-based SSH access).

> These updates ensure that the tools can communicate correctly and execute remote tasks securely.

---

### 🚀 Available Ansible Playbooks

#### Server Run Playbook

```bash
ansible-playbook serverrun-playbook.yml --extra-vars "server=server2"
```

#### JamRun Playbook

```bash
ansible-playbook jamrun-playbook.yml --extra-vars "server=server2 file=jt1.jxe app='--app=DEMO'"
```

#### JamList Playbook

```bash
ansible-playbook jamlist-playbook.yml --extra-vars "server=server2"
```

#### JamTerm Playbook (Without tmux session)

```bash
ansible-playbook jamterm-playbook.yml --extra-vars "server=server2"
```

#### JamTerm Playbook (With tmux session)

```bash
ansible-playbook jamterm-playbook.yml --extra-vars "server=server2 tmux=u-1001-dev-112-c"
```

#### JamKill Playbook

```bash
ansible-playbook jamkill-playbook.yml --extra-vars "server=server2"
```

#### Server Kill Playbook

```bash
ansible-playbook serverkill-playbook.yml --extra-vars "server=server2"

Programs written in JAMScript have worked in Linux (Arch, Ubuntu, Raspbian) and Mac OS.
2 changes: 1 addition & 1 deletion deps/tinycbor
Submodule tinycbor updated from e5414c to 26c63e
2 changes: 1 addition & 1 deletion install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -149,7 +149,7 @@ cd $SCRIPT_DIR
git submodule update --init --recursive lib/

#install tinycbor
cd $SCRIPT_DIR/deps/tinycbor
cd $SCRIPT_DIR/deps
git clone https://github.com/intel/tinycbor
cd tinycbor
make
Expand Down
2 changes: 1 addition & 1 deletion mainCompiler.js
Original file line number Diff line number Diff line change
Expand Up @@ -210,7 +210,7 @@ function nativeCompile(code, cargs, userIncludes, userLinkerFlags) {
fs.writeFileSync(`${tmpDir}/jamout.c`, code);

try {
var command = `clang -g ${tmpDir}/jamout.c -o ${tmpDir}/a.out -I/usr/local/include -I${homeDir}/.jamruns/clib/include -I${homeDir}/.jamruns/clib/src ${options} -pthread -ltinycbor -lmosquitto -lhiredis -levent ${userLinkerFlags.join(" ")} ${homeDir}/.jamruns/clib/libjam.a ${homeDir}/.jamruns/jamhome/deps/mujs2/build/release/libmujs.a -L/usr/local/lib ${includes}`;
var command = `clang -g ${tmpDir}/jamout.c -o ${tmpDir}/a.out -I/usr/local/include -I${homeDir}/.jamruns/clib/include -I${homeDir}/.jamruns/clib/src ${options} -pthread -ltinycbor -lmosquitto -lhiredis -levent ${userLinkerFlags.join(" ")} ${homeDir}/.jamruns/clib/libjam.a ${homeDir}/.jamruns/jamhome/deps/mujs2/build/release/libmujs.a ${homeDir}/.jamruns/jamhome/deps/tinycbor/lib/libtinycbor.a ${includes}`;

if (args.verbosity) console.log("[C] Compiling code...");
if (cargs.verbosity == 2) {
Expand Down
10 changes: 7 additions & 3 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -24,22 +24,26 @@
},
"dependencies": {
"cbor-x": "1.5.4",
"command-line-args": ">= 3.0.5",
"chokidar": "^3.6.0",
"command-line-args": "^5.2.1",
"command-line-usage": "^6.1.0",
"express": "^4.18.2",
"ioredis": "^5.3.2",
"jszip": "^3.10.1",
"lodash": "^4.17.21",
"mercator-projection": "^0.0.2",
"mqtt": "=4.3.7",
"node-fetch": "^3.3.2",
"node-localstorage": ">= 1.3.0",
"node-ssh": "^13.1.0",
"ohm-js": "17.1.0",
"ohm-js": "^17.1.0",
"pino": "^8.15.0",
"ps-node": "^0.1.6",
"random-js": "^2.1.0",
"redis": "^2.8.0",
"toml": "3.0.0",
"ws": "^8.13.0"
"ws": "^8.13.0",
"zx": "^8.3.0"
},
"engines": {
"node": ">= 12.22",
Expand Down
69 changes: 57 additions & 12 deletions scripts/install/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,36 +1,81 @@
FROM ubuntu

# Install OpenSSH server and sudo
RUN apt update && apt install -y openssh-server sudo

RUN sed -i 's/PermitRootLogin prohibit-password/PermitRootLogin yes/' /etc/ssh/sshd_config

RUN apt-get update && apt-get install -y --no-install-recommends apt-utils && \
apt-get install -y -q wget \
build-essential \
python \
python3 \
unzip \
libbsd-dev \
git \
sudo \
vim \
curl \
libssl-dev \
cmake \
make \
clang \
iproute2 \
net-tools \
iputils-ping \
unzip \
iputils-ping

RUN apt-get install -y -q mosquitto \
mosquitto-clients \
libmosquitto-dev

RUN apt-get install -y -q redis-server \
redis-tools \
libhiredis-dev

RUN apt-get install -y -q libevent-dev \
tmux \
inotify-tools

RUN useradd -m admin && echo "admin:admin" | chpasswd && adduser admin sudo
RUN echo "admin ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers
USER admin
WORKDIR /home/admin
RUN wget https://deb.nodesource.com/setup_13.x && chmod +x ./setup_13.x && echo admin | sudo -S ./setup_13.x && \
RUN wget https://deb.nodesource.com/setup_20.x && chmod +x ./setup_20.x && echo admin | sudo -S ./setup_20.x && \
sudo apt-get install -y -q nodejs

RUN git clone https://github.com/citelab/JAMScript
RUN git clone --branch master https://github.com/JeanKa25/JAMScript && cd JAMScript/tools && chmod 777 *


RUN mkdir ~/.npm-global && npm config set prefix '~/.npm-global'
WORKDIR /home/admin/JAMScript/deps
RUN git clone https://github.com/intel/tinycbor
WORKDIR /home/admin/JAMScript/deps/tinycbor
RUN make
RUN sudo make install

WORKDIR /home/admin/JAMScript/deps/mujs2
RUN make
RUN sudo make install

WORKDIR /home/admin/JAMScript
RUN npm run link
ENV PATH="/home/admin/.npm-global/bin:$PATH"
RUN mkdir /home/admin/temp
WORKDIR /home/admin/temp
RUN npm install
RUN sudo npm install zx -g

WORKDIR /home/admin/JAMScript/lib/cside
RUN make archive

WORKDIR /home/admin
RUN mkdir .jamruns
WORKDIR /home/admin/.jamruns
RUN ln -s /home/admin/JAMScript/lib/cside clib
RUN ln -s /home/admin/JAMScript jamhome
RUN ln -s /home/admin/JAMScript/lib/jside node_modules

WORKDIR /home/admin
RUN mkdir .npm-global
RUN mkdir .npm-global/bin

ENV PATH="$PATH:/home/admin/JAMScript/tools:/home/admin/.npm-global/bin"

# Expose the SSH port
EXPOSE 22

CMD sudo chmod -R a+Xr /etc/avahi && sudo sed -i "s|rlimit-nproc=3|#rlimit-nproc=3 |g" /etc/avahi/avahi-daemon.conf && sudo chmod -R a+Xr /var/run && cd /var/run/ && sudo rm -rf dbus && sudo mkdir dbus && sudo dbus-daemon --system && sudo avahi-daemon
# Start SSH service and keep the container running
ENTRYPOINT ["bash", "-c", "sudo service ssh start && exec bash"]
25 changes: 25 additions & 0 deletions tests/C2J_NoReturnResults/jt1-2.c
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
jasync localme(int c, char* s) {
jarray int num[5] = {1, 2, 3, 4, 5};
while(1) {
// jsys.sleep(2000000);
//jsys.dontyield();
you(s, &num);
// wprintf(L"############-->>> Hello ME %d... %s\n", c, s);
}
}

jasync localyou(int c, char* s) {
jarray int num[5] = {1, 2, 3, 4, 5};
while(1) {
jsys.sleep(1000000);
wprintf(L"############-->>> Hello YOU %d, %s\n", c, s);
you(s, &num);
}
}

int main(int argc, char* argv[]) {
wprintf(L"folks we are printing with wide characters\n");
localme(FLT_MANT_DIG, "cxxxxyyyy");
// localyou(hello(), "a-message-for-j: " BAZINGA);
return 0;
}
Loading