Merge pull request #158 from guydavis/integration

v0.5.0

.github/workflows/develop.yaml

@@ -39,8 +39,8 @@ jobs:
           platforms: linux/amd64
           push: true
           build-args: |
-            "CHIA_BRANCH=1.1.7"
-            "PATCH_CHIAPOS=true"
+            "CHIA_BRANCH=1.2.0"
+            "FLAX_BRANCH=main"
           tags: |
             ${{ secrets.DOCKERHUB_USERNAME }}/machinaris:develop
             ghcr.io/${{ secrets.DOCKERHUB_USERNAME }}/machinaris:develop

.github/workflows/main.yaml

@@ -40,8 +40,8 @@ jobs:
           platforms: linux/amd64
           push: true
           build-args: |
-            "CHIA_BRANCH=1.1.7"
-            "PATCH_CHIAPOS=true"
+            "CHIA_BRANCH=1.2.0"
+            "FLAX_BRANCH=main"
           tags: |
             ${{ secrets.DOCKERHUB_USERNAME }}/machinaris:latest
             ${{ secrets.DOCKERHUB_USERNAME }}/machinaris:v${{ github.event.inputs.version }}

.github/workflows/test.yaml

@@ -40,8 +40,8 @@ jobs:
           platforms: linux/amd64
           push: true
           build-args: |
-            "CHIA_BRANCH=1.1.7"
-            "PATCH_CHIAPOS=true"
+            "CHIA_BRANCH=1.2.0"
+            "FLAX_BRANCH=main"
           tags: |
             ${{ secrets.DOCKERHUB_USERNAME }}/machinaris:test
             ghcr.io/${{ secrets.DOCKERHUB_USERNAME }}/machinaris:test

CHANGELOG.md

@@ -2,6 +2,12 @@
 
 All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.5.0] - 2021-07-09
+
+- Support for [official Chia pools](https://github.com/guydavis/machinaris/issues/131). Chia and Madmax plotters can create portable plots.
+- Plotting and farming on the [Flax Network](https://github.com/guydavis/machinaris/issues/105). Enabled by default, but can be [disabled](https://github.com/guydavis/machinaris/wiki/Flax#optional-to-disable).
+
+
 ## [0.4.0] - 2021-06-25
 
 - Support for [Madmax plotter](https://github.com/madMAx43v3r/chia-plotter), in addition to official [Chia plotter](https://github.com/Chia-Network/chia-blockchain).

CREDITS.md

@@ -19,8 +19,9 @@ A huge thank-you to the great teams/devs behind these projects, being used by Ma
 * [Boostrap Sidebars](https://dev.to/codeply/bootstrap-5-sidebar-examples-38pb): Used for sidebar menu layout.
 * [Toplevel](https://github.com/and-semakin/marshmallow-toplevel): Used for list of objects sent to REST API.
 
-## Target Platforms
-* [Unraid](https://selfhosters.net/docker/templating/templating/): Guide to creating an [application template](https://github.com/guydavis/machinaris-unraid) for Unraid.
+## Blockchain Forks
+* [Flax](https://github.com/Flax-Network/flax-blockchain)
+* [Flaxdog](https://github.com/langhorst/flaxdog)
 
 ## Testers and Developers
 
@@ -46,6 +47,11 @@ A big thanks to all that contributed with dev and test including:
 * ServantDad
 * efnats
 * myusuf3
+* kahn2k
+* barkcollar
+* dartec
+* elexx
+
 
 ## Trademark Notice
 CHIA NETWORK INC, CHIA™, the CHIA BLOCKCHAIN™, the CHIA PROTOCOL™, CHIALISP™ and the “leaf Logo” (including the leaf logo alone when it refers to or indicates Chia), are trademarks or registered trademarks of Chia Network, Inc., a Delaware corporation.  *There is no affliation between this Machinaris project and the main Chia Network project.*

README.md

@@ -1,6 +1,6 @@
 # machinaris
 
-A Docker image for plotting and farming the Chia™ cryptocurrency on [one computer](https://github.com/guydavis/machinaris/wiki/Docker) or across [many](https://github.com/guydavis/machinaris/wiki/Workers).
+A Docker image for plotting and farming the Chia™ cryptocurrency on [one computer](https://github.com/guydavis/machinaris/wiki/Docker) or across [many](https://github.com/guydavis/machinaris/wiki/Workers). Now with [official Pool support](https://github.com/guydavis/machinaris/wiki/Pooling)!
 
 ![Home](https://raw.githubusercontent.com/guydavis/machinaris-unraid/master/docs/img/machinaris_home.png)
 
@@ -18,6 +18,8 @@ Machinaris bundles the [latest Chia™ version](https://github.com/Chia-Network/
 
 ![Farming](https://raw.githubusercontent.com/guydavis/machinaris-unraid/master/docs/img/machinaris_farming.png)
 
+Machinaris also optionally farms your plots to the [Flax Network](https://flaxnetwork.org/) too!
+
 ## Alerts
 
 [Chiadog](https://github.com/martomi/chiadog) provides monitoring of the log files, ensuring you get notified when important events occur across your farm:

VERSION

@@ -1 +1 @@
-0.4.0
+0.5.0

api/__init__.py

@@ -35,27 +35,3 @@ def set_sqlite_pragma(dbapi_connection, connection_record):
 
 api = extensions.create_api(app)
 views.register_blueprints(api)
-
-#
-# Notes about using flask-migrate to manage schema:
-# From then on as developer, modify models, then run this to generate a new migration and commit it.
-# cd /code/machinaris/api
-# FLASK_APP=__init__.py flask db migrate -> Creates migration based on current model.
-#
-# Then in scripts/setup_databases.sh, this is run on each launch
-# cd /machinaris/api
-# FLASK_APP=__init__.py flask db upgrade -> Applies migrations against old db
-#
-#
-# Notes about the initial setup:
-#
-# To create very first migration, point to empty sqlite db, by putting these default_settings.py
-#SQLALCHEMY_DATABASE_URI = 'sqlite:///'
-#SQLALCHEMY_BINDS = {
-#    'stats':      'sqlite:///',
-#    'chiadog':    'sqlite:///',
-#}
-# cd /code/machinaris/api
-# FLASK_APP=__init__.py flask db init --multidb
-# FLASK_APP=__init__.py flask db migrate
-# 

api/commands/chia_cli.py

@@ -24,13 +24,22 @@
 from api.models import chia
 
 CHIA_BINARY = '/chia-blockchain/venv/bin/chia'
+FLAX_BINARY = '/flax-blockchain/venv/bin/flax'
 
 # When reading tail of chia plots check output, limit to this many lines
 MAX_LOG_LINES = 2000
 
-def load_farm_summary():
+def get_binary(blockchain):
+    if blockchain == "chia":
+        return CHIA_BINARY
+    if blockchain == "flax":
+        return FLAX_BINARY
+    raise Exception("Invalid blockchain: ".format(blockchain))
+
+def load_farm_summary(blockchain):
+    chia_binary = get_binary(blockchain)
     if globals.farming_enabled(): # Load from chia farm summary
-        proc = Popen("{0} farm summary".format(CHIA_BINARY), stdout=PIPE, stderr=PIPE, shell=True)
+        proc = Popen("{0} farm summary".format(chia_binary), stdout=PIPE, stderr=PIPE, shell=True)
         try:
             outs, errs = proc.communicate(timeout=90)
         except TimeoutExpired:
@@ -51,7 +60,7 @@ def load_plots_farming():
         try:
             entries = (os.path.join(dir_path, file_name) for file_name in os.listdir(dir_path))
             entries = ((os.stat(path), path) for path in entries)
-            entries = ((stat[ST_CTIME], stat[ST_SIZE], path) for stat, path in entries if S_ISREG(stat[ST_MODE]))
+            entries = ((stat[ST_MTIME], stat[ST_SIZE], path) for stat, path in entries if S_ISREG(stat[ST_MODE]))
             all_entries.extend(entries)
         except:
             app.logger.info("Failed to list files at {0}".format(dir_path))
@@ -61,16 +70,16 @@ def load_plots_farming():
     plots_farming = chia.FarmPlots(all_entries)
     return plots_farming
 
-def load_config():
-    return open('/root/.chia/mainnet/config/config.yaml','r').read()
+def load_config(blockchain):
+    return open('/root/.{0}/mainnet/config/config.yaml'.format(blockchain),'r').read()
 
-def save_config(config):
+def save_config(config, blockchain):
     try:
         # Validate the YAML first
         yaml.safe_load(config)
         # Save a copy of the old config file
-        src="/root/.chia/mainnet/config/config.yaml"
-        dst="/root/.chia/mainnet/config/config."+time.strftime("%Y%m%d-%H%M%S")+".yaml"
+        src="/root/.{0}/mainnet/config/config.yaml".format(blockchain)
+        dst="/root/.{0}/mainnet/config/config.".format(blockchain) + time.strftime("%Y%m%d-%H%M%S")+".yaml"
         shutil.copy(src,dst)
         # Now save the new contents to main config file
         with open(src, 'w') as writer:
@@ -79,12 +88,13 @@ def save_config(config):
         app.logger.info(traceback.format_exc())
         raise Exception('Updated config.yaml failed validation!\n' + str(ex))
     else:
-        # TODO restart chia services
+        # TODO restart chia or flax services
         pass
 
-def load_wallet_show():
+def load_wallet_show(blockchain):
+    chia_binary = get_binary(blockchain)
     wallet_show = ""
-    child = pexpect.spawn("{0} wallet show".format(CHIA_BINARY))
+    child = pexpect.spawn("{0} wallet show".format(chia_binary))
     wallet_index = 1
     while True:
         i = child.expect(["Wallet height:.*\r\n", "Choose wallet key:.*\r\n", "No online backup file found.*\r\n"])
@@ -103,8 +113,31 @@ def load_wallet_show():
             wallet_show += "ERROR:\n" + child.after.decode("utf-8") + child.before.decode("utf-8") + child.read().decode("utf-8")
     return chia.Wallet(wallet_show)
 
-def load_blockchain_show():
-    proc = Popen("{0} show --state".format(CHIA_BINARY), stdout=PIPE, stderr=PIPE, shell=True)
+def load_plotnft_show(blockchain):
+    chia_binary = get_binary(blockchain)
+    wallet_show = ""
+    child = pexpect.spawn("{0} plotnft show".format(chia_binary))
+    wallet_index = 1
+    while True:
+        i = child.expect(["Wallet height:.*\r\n", "Choose wallet key:.*\r\n", "No online backup file found.*\r\n"])
+        if i == 0:
+            app.logger.debug("wallet show returned 'Wallet height...' so collecting details.")
+            wallet_show += child.after.decode("utf-8") + child.before.decode("utf-8") + child.read().decode("utf-8")
+            break
+        elif i == 1:
+            app.logger.debug("wallet show got index prompt so selecting #{0}".format(wallet_index))
+            child.sendline("{0}".format(wallet_index))
+            wallet_index += 1
+        elif i == 2:
+            child.sendline("S")
+        else:
+            app.logger.debug("pexpect returned {0}".format(i))
+            wallet_show += "ERROR:\n" + child.after.decode("utf-8") + child.before.decode("utf-8") + child.read().decode("utf-8")
+    return chia.Wallet(wallet_show)
+
+def load_blockchain_show(blockchain):
+    chia_binary = get_binary(blockchain)
+    proc = Popen("{0} show --state".format(chia_binary), stdout=PIPE, stderr=PIPE, shell=True)
     try:
         outs, errs = proc.communicate(timeout=90)
     except TimeoutExpired:
@@ -115,8 +148,9 @@ def load_blockchain_show():
         abort(500, description=errs.decode('utf-8'))
     return chia.Blockchain(outs.decode('utf-8').splitlines())
 
-def load_connections_show():
-    proc = Popen("{0} show --connections".format(CHIA_BINARY), stdout=PIPE, stderr=PIPE, shell=True)
+def load_connections_show(blockchain):
+    chia_binary = get_binary(blockchain)
+    proc = Popen("{0} show --connections".format(chia_binary), stdout=PIPE, stderr=PIPE, shell=True)
     try:
         outs, errs = proc.communicate(timeout=90)
     except TimeoutExpired:
@@ -127,14 +161,15 @@ def load_connections_show():
         abort(500, description=errs.decode('utf-8'))
     return chia.Connections(outs.decode('utf-8').splitlines())
 
-def add_connection(connection):
+def add_connection(connection, blockchain):
+    chia_binary = get_binary(blockchain)
     try:
         hostname,port = connection.split(':')
         if socket.gethostbyname(hostname) == hostname:
             app.logger.debug('{} is a valid IP address'.format(hostname))
         elif socket.gethostbyname(hostname) != hostname:
             app.logger.debug('{} is a valid hostname'.format(hostname))
-        proc = Popen("{0} show --add-connection {1}".format(CHIA_BINARY, connection), stdout=PIPE, stderr=PIPE, shell=True)
+        proc = Popen("{0} show --add-connection {1}".format(chia_binary, connection), stdout=PIPE, stderr=PIPE, shell=True)
         try:
             outs, errs = proc.communicate(timeout=90)
         except TimeoutExpired:
@@ -209,7 +244,14 @@ def generate_key(key_path):
                 return False
         flash('Welcome! A new key has been generated at {0}. Keep it secret! Keep it safe!'.format(key_path), 'success')
         flash('{0}'.format(" ".join(mnemonic_words)), 'info')
-    proc = Popen("{0} start farmer".format(CHIA_BINARY), stdout=PIPE, stderr=PIPE, shell=True)
+        start_farmer('chia')
+        if globals.flax_enabled():
+            # TODO 'flax keys add' the new key
+            start_farmer('flax')
+
+def start_farmer(blockchain):
+    chia_binary = get_binary(blockchain)
+    proc = Popen("{0} start farmer".format(chia_binary), stdout=PIPE, stderr=PIPE, shell=True)
     try:
         outs, errs = proc.communicate(timeout=90)
     except TimeoutExpired:
@@ -226,9 +268,10 @@ def generate_key(key_path):
         return False
     return True
 
-def remove_connection(node_id, ip):
+def remove_connection(node_id, ip, blockchain):
+    chia_binary = get_binary(blockchain)
     try:
-        proc = Popen("{0} show --remove-connection {1}".format(CHIA_BINARY, node_id), stdout=PIPE, stderr=PIPE, shell=True)
+        proc = Popen("{0} show --remove-connection {1}".format(chia_binary, node_id), stdout=PIPE, stderr=PIPE, shell=True)
         try:
             outs, errs = proc.communicate(timeout=90)
         except TimeoutExpired:
@@ -264,7 +307,7 @@ def check_plots(first_load):
             app.logger.info(traceback.format_exc())
             return 'Failed to start plots check job!'
         else:
-            return "Starting chia plots check at " + datetime.datetime.now().strftime("%Y-%m-%d %H:%M")
+            return "Starting plots check at " + datetime.datetime.now().strftime("%Y-%m-%d %H:%M")
     else:
         class_escape = re.compile(r' chia.plotting.(\w+)(\s+): ')
         ansi_escape = re.compile(r'\x1B(?:[@-Z\\-_]|\[[0-?]*[ -/]*[@-~])')

api/commands/chiadog_cli.py

@@ -18,16 +18,16 @@
 from api.models import chiadog
 from api import app
 
-def load_config():
-    return open('/root/.chia/chiadog/config.yaml','r').read()
+def load_config(blockchain):
+    return open('/root/.chia/{0}dog/config.yaml'.format(blockchain),'r').read()
 
-def save_config(config):
+def save_config(config, blockchain):
     try:
         # Validate the YAML first
         yaml.safe_load(config)
         # Save a copy of the old config file
-        src="/root/.chia/chiadog/config.yaml"
-        dst="/root/.chia/chiadog/config.yaml."+time.strftime("%Y%m%d-%H%M%S")+".yaml"
+        src='/root/.chia/{0}dog/config.yaml'.format(blockchain)
+        dst='/root/.chia/{0}dog/config.yaml'.format(blockchain)+time.strftime("%Y%m%d-%H%M%S")+".yaml"
         shutil.copy(src,dst)
         # Now save the new contents to main config file
         with open(src, 'w') as writer:
@@ -36,13 +36,13 @@ def save_config(config):
         app.logger.info(traceback.format_exc())
         raise Exception('Updated config.yaml failed validation!\n' + str(ex))
     else:
-        if get_chiadog_pid():
-            stop_chiadog()
-            start_chiadog()
+        if get_chiadog_pid(blockchain):
+            stop_chiadog(blockchain)
+            start_chiadog(blockchain)
 
-def get_chiadog_pid():
+def get_chiadog_pid(blockchain):
     for proc in psutil.process_iter(['pid', 'name', 'cmdline']):
-        if proc.info['name'] == 'python3' and '/root/.chia/chiadog/config.yaml' in proc.info['cmdline']:
+        if proc.info['name'] == 'python3' and '/root/.chia/{0}dog/config.yaml'.format(blockchain) in proc.info['cmdline']:
             return proc.info['pid']
     return None
 
@@ -67,21 +67,25 @@ def dispatch_action(job):
         raise Exception("Unsupported action {0} for monitoring.".format(action))
 
 def start_chiadog():
-    #app.logger.info("Starting Chiadog monitoring....")
-    try:
-        workdir = "/chiadog"
-        configfile = "/root/.chia/chiadog/config.yaml"
-        logfile = "/root/.chia/chiadog/logs/chiadog.log"
-        proc = Popen("nohup /chia-blockchain/venv/bin/python3 -u main.py --config {0} >> {1} 2>&1 &".format(configfile, logfile), \
-            shell=True, universal_newlines=True, stdout=None, stderr=None, cwd="/chiadog")
-    except:
-        app.logger.info('Failed to start Chiadog monitoring!')
-        app.logger.info(traceback.format_exc())
+    #app.logger.info("Starting monitoring....")
+    blockchains = [ b.strip() for b in os.environ['blockchains'].split(',') ]
+    for blockchain in blockchains:
+        try:
+            workdir = "/{0}dog".format(blockchain)
+            configfile = "/root/.chia/{0}dog/config.yaml".format(blockchain)
+            logfile = "/root/.chia/{0}dog/logs/{0}dog.log".format(blockchain)
+            proc = Popen("nohup /{0}-blockchain/venv/bin/python3 -u main.py --config {1} >> {2} 2>&1 &".format(blockchain, configfile, logfile), \
+                shell=True, universal_newlines=True, stdout=None, stderr=None, cwd="/{0}dog".format(blockchain))
+        except:
+            app.logger.info('Failed to start monitoring!')
+            app.logger.info(traceback.format_exc())
 
 def stop_chiadog():
-    #app.logger.info("Stopping Chiadog monitoring....")
-    try:
-        os.kill(get_chiadog_pid(), signal.SIGTERM)
-    except:
-        app.logger.info('Failed to stop Chiadog monitoring!')
-        app.logger.info(traceback.format_exc())
+    #app.logger.info("Stopping monitoring....")
+    blockchains = [ b.strip() for b in os.environ['blockchains'].split(',') ]
+    for blockchain in blockchains:
+        try:
+            os.kill(get_chiadog_pid(blockchain), signal.SIGTERM)
+        except:
+            app.logger.info('Failed to stop monitoring!')
+            app.logger.info(traceback.format_exc())