Merge branch 'dev'

* dev:
  Documentation update.
  Clear cache
  EWS: Change first chat to File helper
  ETM: Update chat assoc logic.
  Fixed typo
  Added chat linking indicator.
  ETM: Group unlink msg bug fix.
  ETM: bug fix
  More more debug msg
  More debug msg
  Added traceback logging for ETM: process_msg
  EWS: Reversing UID Priority order.
  EWS: Update priority of UID detection
  no message
  Fixed a typo
  Added log filtering
  Add detailed logging, and async msg processing for ETM.
  Try to count for queue time consumption.
  EWS: WeChat disconnection system notice.

Author: blueset

.gitignore

@@ -94,28 +94,7 @@ ENV/
 
 # Covers JetBrains IDEs: IntelliJ, RubyMine, PhpStorm, AppCode, PyCharm, CLion, Android Studio and Webstorm
 # Reference: https://intellij-support.jetbrains.com/hc/en-us/articles/206544839
-
-# User-specific stuff:
-.idea/workspace.xml
-.idea/tasks.xml
-.idea/dictionaries
-.idea/vcs.xml
-.idea/jsLibraryMappings.xml
-
-# Sensitive or high-churn files:
-.idea/dataSources.ids
-.idea/dataSources.xml
-.idea/dataSources.local.xml
-.idea/sqlDataSources.xml
-.idea/dynamic.xml
-.idea/uiDesigner.xml
-
-# Gradle:
-.idea/gradle.xml
-.idea/libraries
-
-# Mongo Explorer plugin:
-.idea/mongoSettings.xml
+.idea
 
 ## File-based project format:
 *.iws

.idea/.name

@@ -1,6 +1,6 @@
 # EH Forwarder Bot
 
-![Python 3.x](https://img.shields.io/badge/Python-3.x-blue.svg)
+![Python ≥ 3.5](https://img.shields.io/badge/Python-≥_3.5-blue.svg)
 [![Gitter](https://img.shields.io/gitter/room/blueset/ehForwarderBot.svg)](https://gitter.im/blueset/ehForwarderBot)
 [![Telegram support group](https://img.shields.io/badge/Chat-on%20Telegram-blue.svg)](https://telegram.me/efbsupport)
 [![license](https://img.shields.io/github/license/blueset/ehforwarderbot.svg)](LICENSE)

.idea/ehForwarderBot.iml

@@ -16,6 +16,7 @@ class MsgType:
 class MsgSource:
     User = "User"
     Group = "Group"
+    System = "System"
 
 
 class TargetType:

.idea/encodings.xml

@@ -32,6 +32,9 @@ Include details about your configuration and environment:
 * **What's the name and version of the OS you're using?**
 * **Which channel caused the problem?** Or did the framework cause it?
 
+!!! attention
+    When submitting your log, please remember to hide your private information.
+
 ## Suggesting enhancements
 
 If you have any suggestions, feel free to raise it up in the issue list. Please try to provide as much details as you can, that includes:

.idea/inspectionProfiles/Project_Default.xml

@@ -1,5 +1,4 @@
 # EFB Telegram Master
-
 EFB Telegram Master (ETM) is a master channel for EFB based on [`python-telegram-bot`](https://python-telegram-bot.org/) and [Telegram Official Bot API](https://core.telegram.org/bots/api).
 
 ## Specs
@@ -18,8 +17,8 @@ pydub
 moviepy
 peewee
 ```
-> **Note**  
-> Please refer to [Pillow documentation](https://pillow.readthedocs.io/en/3.0.x/installation.html) for details on installing Pillow.
+!!! note
+    Please refer to [Pillow documentation](https://pillow.readthedocs.io/en/3.0.x/installation.html) for details on installing Pillow.
 
 #### Non-python dependencies
 ```
@@ -29,9 +28,11 @@ ffmpeg
 and all other required by Pillow.
 
 ### Configuration
+* Create a bot with [@BotFather](https://telegram.me/botfather). [Related tutorial](https://core.telegram.org/bots#6-botfather).
+* Set `/setjoingroups` to `Enable`, `/setprivacy` to `Disable`.
 * Copy `eh_telegram_master` to "plugins" directory,  
   _May not be necessary as it's a built-in plugin of EFB_
-* Set `master = "plugins.eh_telegram_master", "TelegramChannel"` in `config.py`
+* Set `master_channel = "plugins.eh_telegram_master", "TelegramChannel"` in `config.py`
 * Give read and write access to `eh_telegram_master` directory
 * Add variable `eh_telegram_master` as follows
 
@@ -48,7 +49,7 @@ eh_telegram_master = {
 }
 ```
 
-* `token`: (string) Telegram Bot token issued by [@BotFather](https://telegram.me/botfather). [Related tutorial](https://core.telegram.org/bots#6-botfather).
+* `token`: (string) Telegram Bot token issued by BotFather.
 * `admins`: (list of int) User IDs of admin.  
   **Only** messages from admins will be processed. All other messages will be discarded.  
   There are several ways obtaining your ID: Send a message to [@JackBot](https://telegram.me/jackbot), [@get_id_bot](https://telegram.me/get_id_bot), [@UserInfoBot](https://telegram.me/userinfobot), or check it with [Plus messenger](http://plusmessenger.org/).  
@@ -73,8 +74,8 @@ No extra action required during start up.
 ## Usage
 At the beginning, messages from all senders will be sent to the user directly, that means every message  will be mixed in the same conversation. By linking a chat, you can redirect messages from a specific sender to an empty group for a more organized conversation.
 
-> **Notice**:  
-In case of multiple admins are assigned, they may all send message on your behalf, but only the 0th admin can receive direct message from the bot.
+!!! note "Notice"  
+    In case of multiple admins are assigned, they may all send message on your behalf, but only the 0th admin can receive direct message from the bot.
 
 ### Link a chat
 1. Create a new group, invite your bot to the group
@@ -83,10 +84,11 @@ In case of multiple admins are assigned, they may all send message on your behal
    _You can also choose to unlink or relink a linked chat from this menu._
 4. Tap "Start" at the bottom of your screen, and you should see a success message: "Chat associated."
 
-> **Notice**:  
-You may introduce other non-ETM admins to the group, however, they:
-> * Can read all messages send from the related remote chat;
-> * May NOT send message on your behalf.
+!!! note "Notice"
+    You may introduce other non-ETM admins to the group, however, they:
+
+    * Can read all messages send from the related remote chat;
+    * May NOT send message on your behalf.
 
 ### Send a message
 #### Send to a linked chat
@@ -107,8 +109,8 @@ To send a message to a non-linked chat, you should "direct reply" to a message o
 
 In a non-linked chat, direct reply will not be delivered to the remote channel, everything else is supported as it does in a linked chat.
 
-> **Tip**  
-When direct replying to voice messages, always reply to the message preceding to it, or the message with the name of sender. Do not reply to the message itself.
+!!! tip
+    When direct replying to voice messages, always reply to the message preceding to it, or the message with the name of sender. Do not reply to the message itself.
 
 #### Chat head
 If you want to send a message to a non-linked chat which has not yet sent you a message, you can ask ETM to generate a "chat head". Chat head works similarly to an incoming message, you can reply to it to send messages to your recipient.
@@ -159,3 +161,6 @@ ko-KR | - | Korean
 nb-NO | - | Norwegian
 it-IT | - | Italian
 sv-SE | - | Swedish
+
+## Known issues
+* In rare cases, some messages may take 20 to 35 minutes to be delivered to user. (Upstream library [python-telegram-bot#364](https://github.com/python-telegram-bot/python-telegram-bot/issues/364))

.idea/inspectionProfiles/profiles_settings.xml

@@ -16,8 +16,8 @@ pymagic
 pillow
 xmltodict
 ```
-> **Note**  
-> Please refer to [Pillow documentation](https://pillow.readthedocs.io/en/3.0.x/installation.html) for details on installing Pillow.
+!!! note
+    Please refer to [Pillow documentation](https://pillow.readthedocs.io/en/3.0.x/installation.html) for details on installing Pillow.
 
 #### Non-python dependencies
 ```
@@ -28,7 +28,7 @@ and all other required by Pillow.
 ### Configuration
 * Copy `eh_wechat_slave.py` to "plugins" directory  
   _May not be necessary as it's a built-in plugin of EFB_
-* Append `("plugins.we_wechat_slave", "WeChatChannel")` to `slaves` dict in `config.py`
+* Append `("plugins.we_wechat_slave", "WeChatChannel")` to `slave_chanels` dict in `config.py`
 * No other configuration is required
 
 ### Start up
@@ -61,7 +61,13 @@ and all other required by Pillow.
   Please remove the `itchat.pkl` file in the EFB root directory, and restart EFB for QR code scanning.
 * **Can I log in 2 WeChat accounts concurrently?**  
   No. The feature is not yet available to EWS.
-* **I want to report a bug.**  
-  **I have some suggestions.**  
-  **Can I submit a pull request?**  
-  All bug reports, suggestions and pull requests are welcomed. Please read through and understand the [Contribution guideline](CONTRIBUTING.md) before submitting.
+
+## Known issues
+* Random disconnection may occur occasionally due to the limit of protocol.
+* Chat linking may be unstable sometime due to the limit of API.
+
+!!! note "Technical detail"
+    WeChat does not provide a stable chat identifier, so hash of the name of a user is used for the chat identifier. This may cause 2 issues:
+
+    * Chat is no longer traceable when its name is changed.
+    * Conflict and mis-delivery may happen when 2 users share the same name.

.idea/misc.xml

@@ -1,6 +1,6 @@
 # `EFBChannel`
 
-`EFBChannel` is a base class for EFB channels. All channel classes, master and slave, should extend this class. It can be defined in either a single file or a module folder with `__init__.py`.
+`EFBChannel` is the base class for EFB channels. All channel classes, master and slave, should extend this class. It can be defined in either a single file or a module folder with `__init__.py`.
 
 ## Attributes
 **channel_name** (str)  
@@ -34,6 +34,7 @@ Methods below are required to be implemented for respective channel types noted,
 Initialize the channel and log in the account (if necessary) with in this method. `super().__init__(queue)` must always be called in the beginning.
 
 Args:
+
 * `queue` (queue.Queue): Global message queue, used for message delivery from slave channels to the master channel.
 * `slaves` (dict): All enabled slave channel objects. Format: `"channel_id": channel_object`.
 
@@ -49,9 +50,11 @@ Returns a `dict` of all extra functions of the slave channel, in the format of `
 This method sends messages from master channel to slave channel. All related media files should be cleared when the message is sent.
 
 Args:
+
 * msg (EFBMsg): The message object.
 
 Raises:
+
 * `EFBChatNotFound`
 * `EFBMessageNotFound`
 * `EFBMessageTypeNotSupported`

.idea/modules.xml

@@ -21,38 +21,36 @@ The variable `slave_channels` is a list of tuples of 2 strings where each of the
 
 A sample of config file is available at `config.sample.py`.
 
-> **Example**
->
-> When enabling the following channels:
-> * Master channel
->     * DemoMaster  
->       Import path: `plugins.eh_demo_master`  
->       Class name: `DemoMasterChannel`
-> * Slave channels
->     * DemoSlave  
->       Import path: `plugins.eh_demo_slave`  
->       Class name: `DemoSlaveChannel`
->     * RandomSlave  
->       Import path: `plugins.eh_random_slave`  
->       Class name: `RandomSlaveChannel`
->     * MyChatSlave  
->       Import path: `plugins.eh_mychat_slave`  
->       Class name: `MyChatSlaveChannel`
->
-> You should have the `master_channel` and `slave_channels` variable defined as follow:
->
-> ```python
-> master_channel = "plugins.eh_demo_master", "DemoMasterChannel"
-> slave_channels = [
->     ("plugins.eh_demo_slave", "DemoSlaveChannel"),
->     ("plugins.eh_random_slave", "RandomSlaveChannel"),
->     ("plugins.eh_mychat_slave", "MyChatSlaveChannel")
-> ]
-> ```
-
-> **Technical details**  
-The "import path" and "class name" of the module is actually used to import the channel class at root level.  
-It can be understood as `from <import_path> import <class_name>`.
+!!! note "Example"
+    When enabling the following channels:
+
+    * Master channel
+        * DemoMaster  
+          Import path: `plugins.eh_demo_master`  
+          Class name: `DemoMasterChannel`
+    * Slave channels
+        * DemoSlave  
+          Import path: `plugins.eh_demo_slave`  
+          Class name: `DemoSlaveChannel`
+        * RandomSlave  
+          Import path: `plugins.eh_random_slave`  
+          Class name: `RandomSlaveChannel`
+        * MyChatSlave  
+          Import path: `plugins.eh_mychat_slave`  
+          Class name: `MyChatSlaveChannel`
+
+    You should have the `master_channel` and `slave_channels` variable defined as follow:
+
+        master_channel = "plugins.eh_demo_master", "DemoMasterChannel"
+        slave_channels = [
+            ("plugins.eh_demo_slave", "DemoSlaveChannel"),
+            ("plugins.eh_random_slave", "RandomSlaveChannel"),
+            ("plugins.eh_mychat_slave", "MyChatSlaveChannel")
+        ]
+
+!!! tip "Technical details"
+    The "import path" and "class name" of the module is actually used to import the channel class at root level.  
+    It can be understood as `from <import_path> import <class_name>`.
 
 ## Permissions
 `storage` directory should be given read and write access for media processing.
@@ -65,8 +63,8 @@ For more details about how to configure your channel, please consult the respect
 ## Get it up and running
 Most of the time, you can just run `python3 main.py -d start`  and it should be ready to go.
 
-> **Run it as a normal process**  
-> Besides, you can still use the classic `python3 main.py` to launch EFB. If you want to keep it running in the background when accessing the machine via SSH, you can use tools like `screen` to prevent it from being terminated during disconnection.
+!!! tip "Run it as a normal process"
+    Besides, you can still use the classic `python3 main.py` to launch EFB. If you want to keep it running in the background when accessing the machine via SSH, you can use tools like `screen` to prevent it from being terminated during disconnection.
 
 However, some channels may require one-time credentials (e.g. Dynamic QR code scanning for WeChat Web Protocol). When you run the module, you may be required to take some actions before the bot goes online.
 

README.md

@@ -1,7 +1,6 @@
 # EH Forwarder Bot
 
-![Python 3.x](https://img.shields.io/badge/Python-3.x-blue.svg)
-[![Gitter](https://img.shields.io/gitter/room/blueset/ehForwarderBot.svg)](https://gitter.im/blueset/ehForwarderBot)
+![Python ≥ 3.5](https://img.shields.io/badge/Python-≥_3.5-blue.svg) [![Gitter](https://img.shields.io/gitter/room/blueset/ehForwarderBot.svg)](https://gitter.im/blueset/ehForwarderBot)
 [![Telegram support group](https://img.shields.io/badge/Chat-on%20Telegram-blue.svg)](https://telegram.me/efbsupport)
 [![license](https://img.shields.io/github/license/blueset/ehforwarderbot.svg)](LICENSE)