update to latest diem core master (#177)

* add back diem submodule
* generate code from latest diem core master and fix tests
* rename identifier hrp
* add dry-run mainnet HRP
* rename all the rest 'libra' to diem
* faucet support creates dd account
* use devnet for testnet until we deploy new master to testnet
* bump up version to 1.0.0

Author: Xiao Li

.gitmodules

@@ -0,0 +1,3 @@
+[submodule "diem"]
+	path = diem
+	url = https://github.com/diem/diem.git

CONTRIBUTING.md

@@ -6,7 +6,7 @@ This project welcomes contributions.
 
 For pull request to be accepted by any Diem projects, a CLA must be [signed](https://www.diem.com/en-us/cla-sign/). You will only need to do this once to work on any of Diem's open source projects.
 
-When submitting a pull request (PR), the `libra-github-bot` will check your submission for a valid CLA. If one is not found, then you will need to [submit](https://www.diem.com/en-us/cla-sign/) an Individual CLA for yourself or a Corporate CLA for your company.
+When submitting a pull request (PR), the `diem-github-bot` will check your submission for a valid CLA. If one is not found, then you will need to [submit](https://www.diem.com/en-us/cla-sign/) an Individual CLA for yourself or a Corporate CLA for your company.
 
 ## Issues
 

Makefile

@@ -35,11 +35,11 @@ diemtypes:
 	"diem/target/debug/generate-transaction-builders" \
 		--language python3 \
 		--module-name stdlib \
-		--with-libra-types "diem/testsuite/generate-format/tests/staged/libra.yaml" \
+		--with-diem-types "diem/testsuite/generate-format/tests/staged/diem.yaml" \
 		--serde-package-name diem \
-		--libra-package-name diem \
+		--diem-package-name diem \
 		--target-source-dir src/diem \
-		--with-custom-libra-code diem-types-ext/*.py \
+		--with-custom-diem-code diem-types-ext/*.py \
 		-- "diem/language/stdlib/compiled/transaction_scripts/abi"
 
 protobuf:

README.md

@@ -1,5 +1,3 @@
-> **Note to readers:** On December 1, 2020, the Libra Association was renamed to Diem Association. The project repos are in the process of being migrated. All projects will remain available for use here until the migration to a new GitHub Organization is complete.
-
 # The official Diem Client SDK for Python.
 
 [Documentation](https://developers.diem.com/docs/python-client-sdk-docs)
@@ -14,13 +12,13 @@ Root module name: `diem`
 
 Sub-modules:
 
-- `jsonrpc`: diem JSON-RPC APIs client and API response types. [SPEC](https://github.com/libra/libra/blob/master/json-rpc/json-rpc-spec.md)
+- `jsonrpc`: diem JSON-RPC APIs client and API response types. [SPEC](https://github.com/diem/diem/blob/master/json-rpc/json-rpc-spec.md)
 - `stdlib`: generated code, move stdlib script utils for constructing transaction script playload.
-- `diem_types`: generated code, Diem on-chain data structure types for encoding and decoding [LCS](https://libra.github.io/libra/lcs/index.html) data.
+- `diem_types`: generated code, Diem on-chain data structure types for encoding and decoding [LCS](https://diem.github.io/diem/lcs/index.html) data.
 - `utils`: utility functions, account address utils, currency code, hashing, hex encoding / decoding, transaction utils.
 - `AuthKey` | `auth_key`: auth key utils
 - `identifier`: Diem Account Identifier and Diem Intent Identifier. [DIP-5](https://dip.diem.com/dip-5/)
-- `txnmetadata`: utils for creating peer to peer transaction metadata. [DIP-4](https://dip.diem.org/dip-4/)
+- `txnmetadata`: utils for creating peer to peer transaction metadata. [DIP-4](https://dip.diem.com/dip-4/)
 - `testnet`: Testnet utility, minting coins, create Testnet client, chain id, Testnet JSON-RPC URL.
 - `LocalAccount` | `local_account`: utility for managing local account keys, generate random local account.
 - `chain_ids`: list of static chain ids
@@ -55,7 +53,7 @@ pip install diem-client-sdk
 
 ## Bugs/Requests
 
-Please use the [GitHub issue tracker](https://github.com/libra/client-sdk-python/issues) to submit bugs or request features.
+Please use the [GitHub issue tracker](https://github.com/diem/client-sdk-python/issues) to submit bugs or request features.
 
 ## Build & Test
 

checklist.md

@@ -28,7 +28,7 @@ This file is a checklist of requirement & technical details for a Diem client SD
   - [x] client tracks latest server response block version and timestamp, raise error when received server response contains stale version / timestamp.
     - [x] last known blockchain version >= response version: when connecting to a cluster of fullnodes, it is possible some fullnodes are behind the head couple versions.
     - [x] last known blockchain timestamp >= response timestamp.
-  - [x] parse and use libra_chain_id, libra_ledger_version and libra_ledger_tiemstamp in the JSONRPC response.
+  - [x] parse and use diem_chain_id, diem_ledger_version and diem_ledger_tiemstamp in the JSONRPC response.
 - [x] Parsing and gen Diem Account Identifier (see [DIP-5][2])
   - bech32 addresses/subaddresses support
 - [x] language specific standard release publish: e.g. java maven central repo, python pip
@@ -88,7 +88,7 @@ See [doc][5] for above concepts.
 
 # Examples
 
-- [x] [p2p transfer examples](https://github.com/libra/dip/blob/master/dips/dip-4.md#transaction-examples)
+- [x] [p2p transfer examples](https://github.com/diem/lip/blob/master/lips/lip-4.md#transaction-examples)
 - [x] refund p2p transfer example
 - [x] create childVASP example
 - [x] Intent identifier encoding, decoding example
@@ -98,10 +98,10 @@ See [doc][5] for above concepts.
 - [ ] Async client
 - [ ] CLI connects to testnet for trying out features.
 
-[1]: https://github.com/libra/libra/blob/master/json-rpc/json-rpc-spec.md "Diem JSON-RPC SPEC"
-[2]: https://github.com/libra/dip/blob/master/dips/dip-5.md "DIP-5"
-[3]: https://github.com/libra/libra/blob/master/language/stdlib/transaction_scripts/doc/peer_to_peer_with_metadata.md "P2P Transafer"
-[4]: https://github.com/libra/libra/tree/master/language/stdlib/transaction_scripts/doc "Move Stdlib scripts"
-[5]: https://github.com/libra/libra/blob/master/client/libra-dev/README.md "Diem Client Dev Doc"
-[6]: https://github.com/libra/libra/blob/master/json-rpc/docs/service_testnet_faucet.md "Faucet service"
-[7]: https://github.com/libra/dip/blob/master/dips/dip-4.md "Transaction Metadata Specification"
+[1]: https://github.com/diem/diem/blob/master/json-rpc/json-rpc-spec.md "Diem JSON-RPC SPEC"
+[2]: https://github.com/diem/lip/blob/master/lips/lip-5.md "Address formatting"
+[3]: https://github.com/diem/diem/blob/master/language/stdlib/transaction_scripts/doc/peer_to_peer_with_metadata.md "P2P Transafer"
+[4]: https://github.com/diem/diem/tree/master/language/stdlib/transaction_scripts/doc "Move Stdlib scripts"
+[5]: https://github.com/diem/diem/blob/master/client/diem-dev/README.md "Diem Client Dev Doc"
+[6]: https://github.com/diem/diem/blob/master/json-rpc/docs/service_testnet_faucet.md "Faucet service"
+[7]: https://github.com/diem/lip/blob/master/lips/lip-4.md "Transaction Metadata Specification"

diem

@@ -8,12 +8,12 @@
 def test_intent_identifier():
     child_vasp_address = "d738a0b9851305dfe1d17707f0841dbc"
     user_sub_address = "9072d012034a880f"
-    currency_code = "Coin1"
+    currency_code = "XUS"
     amount = 10_000_000
 
     account_id = identifier.encode_account(
-        child_vasp_address, utils.sub_address(user_sub_address), identifier.TLB  # testnet HRP
+        child_vasp_address, utils.sub_address(user_sub_address), identifier.TDM  # testnet HRP
     )
     intent_id = identifier.encode_intent(account_id, currency_code, amount)
 
-    assert intent_id == "diem://tlb1p6uu2pwv9zvzalcw3wurlppqahjg895qjqd9gsrc5pe4ul?c=Coin1&am=10000000"
+    assert intent_id == "diem://tdm1p6uu2pwv9zvzalcw3wurlppqahjg895qjqd9gsrcr9dqh8?c=XUS&am=10000000"

examples/intent_identifier.py

@@ -49,7 +49,7 @@ def test_non_custodial_to_custodial():
 
     intent_id = receiver_custodial.payment(user_id=0, amount=1_000_000)
 
-    intent = identifier.decode_intent(intent_id, identifier.TLB)
+    intent = identifier.decode_intent(intent_id, identifier.TDM)
 
     script = stdlib.encode_peer_to_peer_with_metadata_script(
         currency=utils.currency_code(intent.currency_code),
@@ -104,7 +104,7 @@ def test_custodial_to_custodial_under_threshold():
 
     intent_id = receiver_custodial.payment(user_id=0, amount=1_000_000)
 
-    intent = identifier.decode_intent(intent_id, identifier.TLB)
+    intent = identifier.decode_intent(intent_id, identifier.TDM)
 
     script = stdlib.encode_peer_to_peer_with_metadata_script(
         currency=utils.currency_code(intent.currency_code),
@@ -134,7 +134,7 @@ def test_custodial_to_custodial_above_threshold():
 
     intent_id = receiver_custodial.payment(user_id=0, amount=2_000_000_000)
 
-    intent = identifier.decode_intent(intent_id, identifier.TLB)
+    intent = identifier.decode_intent(intent_id, identifier.TDM)
 
     sender = sender_custodial.available_child_vasp()
     # sender & receiver communicate by off chain APIs

examples/p2p_transfer.py

@@ -21,7 +21,7 @@ def test_refund_transaction_of_custodial_to_custodial_under_threshold():
 
     # create a payment transaction
     intent_id = receiver_custodial.payment(user_id=0, amount=1_000_000)
-    intent = identifier.decode_intent(intent_id, identifier.TLB)
+    intent = identifier.decode_intent(intent_id, identifier.TDM)
 
     receiver_address = utils.account_address(intent.account_address)
     script = stdlib.encode_peer_to_peer_with_metadata_script(

examples/refund.py

@@ -75,7 +75,7 @@ def add_user(self):
 
     def payment(self, user_id: int, amount: int) -> str:
         account_id = identifier.encode_account(
-            self._children[0].account_address, self._users[user_id], identifier.TLB  # testnet HRP
+            self._children[0].account_address, self._users[user_id], identifier.TDM  # testnet HRP
         )
         return identifier.encode_intent(account_id, testnet.TEST_CURRENCY_CODE, amount)
 

examples/stubs.py

@@ -10,12 +10,12 @@
 
 setup(
     name="diem",
-    version="0.6.2020120202", # bump up version for release, format 0.X.YYYYMMDDNN
+    version="1.0.0",
     description="The Python Client SDK for Diem",
     long_description=long_description,
     long_description_content_type='text/markdown',
     license="Apache-2.0",
-    url="https://github.com/libra/client-sdk-python",
+    url="https://github.com/diem/client-sdk-python",
     # author="Diem Open Source",
     python_requires=">=3.7", # requires dataclasses
     packages=["diem"],

setup.py

@@ -1,7 +1,7 @@
 # Copyright (c) The Diem Core Contributors
 # SPDX-License-Identifier: Apache-2.0
 
-"""Python client sdk library for the [Diem](https://libra.org) blockchain network."""
+"""Python client SDK library for the [Diem](https://diem.com) blockchain network."""
 
 from .utils import InvalidAccountAddressError, InvalidSubAddressError
 from .auth_key import AuthKey

src/diem/__init__.py

@@ -19,4 +19,4 @@
 TESTNET: ChainId = ChainId.from_int(2)
 DEVNET: ChainId = ChainId.from_int(3)
 TESTING: ChainId = ChainId.from_int(4)
-PREMAINNET: ChainId = ChainId.from_int(19)
+PREMAINNET: ChainId = ChainId.from_int(20)

src/diem/chain_ids.py

@@ -26,16 +26,17 @@
 from .bech32 import bech32_address_encode, bech32_address_decode, Bech32Error
 from .subaddress import DIEM_SUBADDRESS_SIZE, DIEM_ZERO_SUBADDRESS, gen_subaddress
 
-LBR = "lbr"  # lbr for mainnet
-TLB = "tlb"  # tlb for testnet
-PLB = "plb"  # plb for premainnet
+DM = "dm"  # mainnet
+TDM = "tdm"  # testnet
+PDM = "pdm"  # premainnet
+DDM = "ddm"  # dry-run mainnet
 
 HRPS: typing.Dict[int, str] = {
-    chain_ids.MAINNET.to_int(): LBR,
-    chain_ids.TESTNET.to_int(): TLB,
-    chain_ids.DEVNET.to_int(): TLB,
-    chain_ids.TESTING.to_int(): TLB,
-    chain_ids.PREMAINNET.to_int(): PLB,
+    chain_ids.MAINNET.to_int(): DM,
+    chain_ids.TESTNET.to_int(): TDM,
+    chain_ids.DEVNET.to_int(): TDM,
+    chain_ids.TESTING.to_int(): TDM,
+    chain_ids.PREMAINNET.to_int(): PDM,
 }
 
 

src/diem/identifier/__init__.py

@@ -19,8 +19,7 @@
 # DIEM constants
 _DIEM_ADDRESS_SIZE = 16  # in bytes
 _DIEM_BECH32_VERSION = 1
-_DIEM_BECH32_SIZE = 50  # in characters
-
+_DIEM_BECH32_SIZE = [50, 49]  # in characters
 
 class Bech32Error(Exception):
     """ Represents an error when creating a Diem address. """
@@ -68,39 +67,41 @@ def bech32_address_decode(expected_hrp: str, bech32: str) -> typing.Tuple[int, b
         A tuple consisiting of the Bech32 version (int), address (16 bytes), subaddress (8 bytes)
     """
     len_bech32 = len(bech32)
+    len_hrp = len(expected_hrp)
+
     # check expected length
-    if len_bech32 != _DIEM_BECH32_SIZE:
+    if len_bech32 not in _DIEM_BECH32_SIZE:
         raise Bech32Error(f"Bech32 size should be {_DIEM_BECH32_SIZE}, but it is: {len_bech32}")
 
     # do not allow mixed case per BIP 173
     if bech32 != bech32.lower() and bech32 != bech32.upper():
         raise Bech32Error(f"Mixed case Bech32 addresses are not allowed, got: {bech32}")
+
     bech32 = bech32.lower()
+    hrp = bech32[:len_hrp]
 
-    if bech32[:3] != expected_hrp:
+    if hrp != expected_hrp:
         raise Bech32Error(
-            f'Wrong Diem address Bech32 human readable part (prefix): requested "{expected_hrp}" but '
-            f'got "{bech32[:3]}"'
+            f'Wrong Diem address Bech32 human readable part (prefix): requested "{expected_hrp}" but ' f'got "{hrp}"'
         )
 
     # check separator
-    if bech32[3] != _BECH32_SEPARATOR:
-        raise Bech32Error(f"Non-expected Bech32 separator: {bech32[3]}")
+    if bech32[len_hrp] != _BECH32_SEPARATOR:
+        raise Bech32Error(f"Non-expected Bech32 separator: {bech32[len_hrp]}")
 
     # check characters after separator in Bech32 alphabet
-    if not all(x in _BECH32_CHARSET for x in bech32[4:]):
+    if not all(x in _BECH32_CHARSET for x in bech32[len_hrp + 1 :]):
         raise Bech32Error(f"Invalid Bech32 characters detected: {bech32}")
-    hrp = bech32[:3]
 
     # version is defined by the index of the Bech32 character after separator
-    address_version = _BECH32_CHARSET.find(bech32[4])
+    address_version = _BECH32_CHARSET.find(bech32[len_hrp + 1])
     # check valid version
     if address_version != _DIEM_BECH32_VERSION:
         raise Bech32Error(f"Version mismatch. Expected {_DIEM_BECH32_VERSION}, " f"but received {address_version}")
 
     # we've already checked that all characters are in the correct alphabet,
     # thus, this will always succeed
-    data = [_BECH32_CHARSET.find(x) for x in bech32[5:]]
+    data = [_BECH32_CHARSET.find(x) for x in bech32[len_hrp + 2 :]]
 
     # check Bech32 checksum
     if not _bech32_verify_checksum(hrp, [address_version] + data):

src/diem/identifier/bech32.py

@@ -16,7 +16,7 @@
 
 ```
 
-See [Diem JSON-RPC API SPEC](https://github.com/libra/libra/blob/master/json-rpc/json-rpc-spec.md) for more details
+See [Diem JSON-RPC API SPEC](https://github.com/diem/diem/blob/master/json-rpc/json-rpc-spec.md) for more details
 
 """
 
@@ -64,7 +64,7 @@
     EVENT_DATA_BURN,
     EVENT_DATA_CANCEL_BURN,
     EVENT_DATA_MINT,
-    EVENT_DATA_TO_LBR_EXCHANGE_RATE_UPDATE,
+    EVENT_DATA_TO_XDX_EXCHANGE_RATE_UPDATE,
     EVENT_DATA_PREBURN,
     EVENT_DATA_RECEIVED_PAYMENT,
     EVENT_DATA_SENT_PAYMENT,
@@ -87,6 +87,6 @@
     TRANSACTION_DATA_USER,
     TRANSACTION_DATA_UNKNOWN,
     # Script#type field values, only set unknown type here,
-    # other types, plese see https://github.com/libra/libra/blob/master/language/stdlib/transaction_scripts/doc/transaction_script_documentation.md for all available script names.
+    # other types, plese see https://github.com/diem/diem/blob/master/language/stdlib/transaction_scripts/doc/transaction_script_documentation.md for all available script names.
     SCRIPT_UNKNOWN,
 )

src/diem/jsonrpc/__init__.py

@@ -172,7 +172,7 @@ def _first_success(self, primary: Future, backup: Future) -> typing.Dict[str, ty
 class Client:
     """Diem JSON-RPC API client
 
-    [SPEC](https://github.com/libra/libra/blob/master/json-rpc/json-rpc-spec.md)
+    [SPEC](https://github.com/diem/diem/blob/master/json-rpc/json-rpc-spec.md)
     """
 
     def __init__(
@@ -301,7 +301,7 @@ def get_metadata(
     ) -> typing.Optional[rpc.Metadata]:
         """get block metadata
 
-        See [JSON-RPC API Doc](https://github.com/libra/libra/blob/master/json-rpc/docs/method_get_metadata.md)
+        See [JSON-RPC API Doc](https://github.com/diem/diem/blob/master/json-rpc/docs/method_get_metadata.md)
         """
 
         params = [int(version)] if version else []
@@ -310,7 +310,7 @@ def get_metadata(
     def get_currencies(self) -> typing.List[rpc.CurrencyInfo]:
         """get currencies
 
-        See [JSON-RPC API Doc](https://github.com/libra/libra/blob/master/json-rpc/docs/method_get_currencies.md)
+        See [JSON-RPC API Doc](https://github.com/diem/diem/blob/master/json-rpc/docs/method_get_currencies.md)
         """
 
         return self.execute("get_currencies", [], _parse_list(lambda: rpc.CurrencyInfo()))
@@ -321,7 +321,7 @@ def get_account(
         """get on-chain account information
 
         Returns None if account not found
-        See [JSON-RPC API Doc](https://github.com/libra/libra/blob/master/json-rpc/docs/method_get_account.md)
+        See [JSON-RPC API Doc](https://github.com/diem/diem/blob/master/json-rpc/docs/method_get_account.md)
         """
 
         address = utils.account_address_hex(account_address)
@@ -337,7 +337,7 @@ def get_account_transaction(
 
         Returns None if transaction is not found
 
-        See [JSON-RPC API Doc](https://github.com/libra/libra/blob/master/json-rpc/docs/method_get_account_transaction.md)
+        See [JSON-RPC API Doc](https://github.com/diem/diem/blob/master/json-rpc/docs/method_get_account_transaction.md)
         """
 
         address = utils.account_address_hex(account_address)
@@ -355,7 +355,7 @@ def get_account_transactions(
 
         Returns empty list if no transactions found
 
-        See [JSON-RPC API Doc](https://github.com/libra/libra/blob/master/json-rpc/docs/method_get_account_transactions.md)
+        See [JSON-RPC API Doc](https://github.com/diem/diem/blob/master/json-rpc/docs/method_get_account_transactions.md)
         """
 
         address = utils.account_address_hex(account_address)
@@ -372,7 +372,7 @@ def get_transactions(
 
         Returns empty list if no transactions found
 
-        See [JSON-RPC API Doc](https://github.com/libra/libra/blob/master/json-rpc/docs/method_get_transactions.md)
+        See [JSON-RPC API Doc](https://github.com/diem/diem/blob/master/json-rpc/docs/method_get_transactions.md)
         """
 
         params = [int(start_version), int(limit), bool(include_events)]
@@ -383,7 +383,7 @@ def get_events(self, event_stream_key: str, start: int, limit: int) -> typing.Li
 
         Returns empty list if no events found
 
-        See [JSON-RPC API Doc](https://github.com/libra/libra/blob/master/json-rpc/docs/method_get_events.md)
+        See [JSON-RPC API Doc](https://github.com/diem/diem/blob/master/json-rpc/docs/method_get_events.md)
         """
 
         params = [event_stream_key, int(start), int(limit)]
@@ -416,7 +416,7 @@ def submit(
 
         Pass in `raise_stale_response=True` to raise StaleResponseError and handle it by yourself.
 
-        See [JSON-RPC API Doc](https://github.com/libra/libra/blob/master/json-rpc/docs/method_submit.md)
+        See [JSON-RPC API Doc](https://github.com/diem/diem/blob/master/json-rpc/docs/method_submit.md)
         """
 
         if isinstance(txn, diem_types.SignedTransaction):
@@ -577,9 +577,9 @@ def _send_http_request(
         # check stable response before check jsonrpc error
         try:
             self.update_last_known_state(
-                json.get("libra_chain_id"),
-                json.get("libra_ledger_version"),
-                json.get("libra_ledger_timestampusec"),
+                json.get("diem_chain_id"),
+                json.get("diem_ledger_version"),
+                json.get("diem_ledger_timestampusec"),
             )
         except StaleResponseError as e:
             if not ignore_stale_response: