-
Notifications
You must be signed in to change notification settings - Fork 22
feat: Implement historical pricing for message cost recalculation #813
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
amalcaraz
wants to merge
3
commits into
main
Choose a base branch
from
angel_feat-message-cost-recalculation-using-historical-prices
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,101 @@ | ||
| #!/usr/bin/env python3 | ||
| """ | ||
| Authentication token generation script for PyAleph. | ||
|
|
||
| This script provides functionality to: | ||
| 1. Generate new ECDSA key pairs | ||
| 2. Create authentication tokens using private keys | ||
| """ | ||
|
|
||
| import argparse | ||
| import getpass | ||
| import sys | ||
| from pathlib import Path | ||
|
|
||
| # Add src directory to Python path for imports | ||
| sys.path.insert(0, str(Path(__file__).parent.parent.parent / 'src')) | ||
|
|
||
| from aleph.toolkit.ecdsa import generate_key_pair, create_auth_token, generate_key_pair_from_private_key | ||
|
|
||
|
|
||
| def generate_new_keypair(): | ||
| """Generate and display a new ECDSA key pair.""" | ||
| print("Generating new ECDSA key pair...") | ||
| private_key, public_key = generate_key_pair() | ||
|
|
||
| print(f"\nPrivate Key (keep secure!): {private_key}") | ||
| print(f"Public Key: {public_key}") | ||
|
|
||
| return private_key, public_key | ||
|
|
||
|
|
||
| def read_keypair(): | ||
| """Create an authentication token from a private key.""" | ||
| print("Enter your private key in hex format:") | ||
| private_key_hex = getpass.getpass("Private Key: ").strip() | ||
|
|
||
| if not private_key_hex: | ||
| print("Error: Private key cannot be empty") | ||
| return None | ||
|
|
||
| private_key, public_key = generate_key_pair_from_private_key(private_key_hex) | ||
|
|
||
| # print(f"\nPrivate Key (keep secure!): {private_key}") | ||
| # print(f"Public Key: {public_key}") | ||
|
|
||
| return private_key, public_key | ||
|
|
||
|
|
||
| def main(): | ||
| parser = argparse.ArgumentParser( | ||
| description="PyAleph Authentication Token Generator", | ||
| formatter_class=argparse.RawDescriptionHelpFormatter, | ||
| epilog=""" | ||
| Examples: | ||
| # Generate a new key pair | ||
| python generate_auth_token.py --generate-keys | ||
|
|
||
| # Create token from existing private key | ||
| python generate_auth_token.py --create-token | ||
|
|
||
| # Do both operations | ||
| python generate_auth_token.py --generate-keys --create-token | ||
| """ | ||
| ) | ||
|
|
||
| parser.add_argument( | ||
| '--generate-keys', '-g', | ||
| action='store_true', | ||
| help='Generate a new ECDSA key pair' | ||
| ) | ||
|
|
||
| parser.add_argument( | ||
| '--create-token', '-t', | ||
| action='store_true', | ||
| help='Create an authentication token from a private key' | ||
| ) | ||
|
|
||
| args = parser.parse_args() | ||
|
|
||
| if not args.generate_keys and not args.create_token: | ||
| parser.print_help() | ||
| return | ||
|
|
||
| private_key = None | ||
|
|
||
| if args.generate_keys: | ||
| private_key, _ = generate_new_keypair() | ||
|
|
||
| if args.create_token: | ||
| if private_key is None: | ||
| private_key, _ = read_keypair() | ||
|
|
||
| try: | ||
| token = create_auth_token(private_key) | ||
| print(f"\nGenerated Token: {token}") | ||
| except Exception as e: | ||
| print(f"Error creating token: {e}") | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| main() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,122 @@ | ||
| """ | ||
| Utility functions for pricing model creation and management. | ||
| """ | ||
|
|
||
| import datetime as dt | ||
| from typing import Dict, List, Union | ||
|
|
||
| from aleph.db.accessors.aggregates import ( | ||
| get_aggregate_elements, | ||
| merge_aggregate_elements, | ||
| ) | ||
| from aleph.db.models import AggregateElementDb | ||
| from aleph.toolkit.constants import ( | ||
| DEFAULT_PRICE_AGGREGATE, | ||
| PRICE_AGGREGATE_KEY, | ||
| PRICE_AGGREGATE_OWNER, | ||
| ) | ||
| from aleph.types.cost import ProductPriceType, ProductPricing | ||
| from aleph.types.db_session import DbSession | ||
|
|
||
|
|
||
| def build_pricing_model_from_aggregate( | ||
| aggregate_content: Dict[Union[ProductPriceType, str], dict] | ||
| ) -> Dict[ProductPriceType, ProductPricing]: | ||
| """ | ||
| Build a complete pricing model from an aggregate content dictionary. | ||
|
|
||
| This function converts the DEFAULT_PRICE_AGGREGATE format or any pricing aggregate | ||
| content into a dictionary of ProductPricing objects that can be used by the cost | ||
| calculation functions. | ||
|
|
||
| Args: | ||
| aggregate_content: Dictionary containing pricing information with ProductPriceType as keys | ||
|
|
||
| Returns: | ||
| Dictionary mapping ProductPriceType to ProductPricing objects | ||
| """ | ||
| pricing_model: Dict[ProductPriceType, ProductPricing] = {} | ||
|
|
||
| for price_type, pricing_data in aggregate_content.items(): | ||
| try: | ||
| price_type = ProductPriceType(price_type) | ||
| pricing_model[price_type] = ProductPricing.from_aggregate( | ||
| price_type, aggregate_content | ||
| ) | ||
| except (KeyError, ValueError) as e: | ||
| # Log the error but continue processing other price types | ||
| import logging | ||
|
|
||
| logger = logging.getLogger(__name__) | ||
| logger.warning(f"Failed to parse pricing for {price_type}: {e}") | ||
|
|
||
| return pricing_model | ||
|
|
||
|
|
||
| def build_default_pricing_model() -> Dict[ProductPriceType, ProductPricing]: | ||
| """ | ||
| Build the default pricing model from DEFAULT_PRICE_AGGREGATE constant. | ||
|
|
||
| Returns: | ||
| Dictionary mapping ProductPriceType to ProductPricing objects | ||
| """ | ||
| return build_pricing_model_from_aggregate(DEFAULT_PRICE_AGGREGATE) | ||
|
|
||
|
|
||
| def get_pricing_aggregate_history(session: DbSession) -> List[AggregateElementDb]: | ||
| """ | ||
| Get all pricing aggregate updates in chronological order. | ||
|
|
||
| Args: | ||
| session: Database session | ||
|
|
||
| Returns: | ||
| List of AggregateElementDb objects ordered by creation_datetime | ||
| """ | ||
| aggregate_elements = get_aggregate_elements( | ||
| session=session, owner=PRICE_AGGREGATE_OWNER, key=PRICE_AGGREGATE_KEY | ||
| ) | ||
| return list(aggregate_elements) | ||
|
|
||
|
|
||
| def get_pricing_timeline( | ||
| session: DbSession, | ||
| ) -> List[tuple[dt.datetime, Dict[ProductPriceType, ProductPricing]]]: | ||
| """ | ||
| Get the complete pricing timeline with timestamps and pricing models. | ||
|
|
||
| This function returns a chronologically ordered list of pricing changes, | ||
| useful for processing messages in chronological order and applying the | ||
| correct pricing at each point in time. | ||
|
|
||
| This properly merges aggregate elements up to each point in time to create | ||
| the cumulative pricing state, similar to how _update_aggregate works. | ||
|
|
||
| Args: | ||
| session: Database session | ||
|
|
||
| Returns: | ||
| List of tuples containing (timestamp, pricing_model) | ||
| """ | ||
| pricing_elements = get_pricing_aggregate_history(session) | ||
|
|
||
| timeline = [] | ||
|
|
||
| # Add default pricing as the initial state | ||
| timeline.append( | ||
| (dt.datetime.min.replace(tzinfo=dt.timezone.utc), build_default_pricing_model()) | ||
| ) | ||
|
|
||
| # Build cumulative pricing models by merging elements up to each timestamp | ||
| elements_so_far = [] | ||
| for element in pricing_elements: | ||
| elements_so_far.append(element) | ||
|
|
||
| # Merge all elements up to this point to get the cumulative state | ||
| merged_content = merge_aggregate_elements(elements_so_far) | ||
|
|
||
| # Build pricing model from the merged content | ||
| pricing_model = build_pricing_model_from_aggregate(merged_content) | ||
| timeline.append((element.creation_datetime, pricing_model)) | ||
|
|
||
| return timeline |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think that we shouldn't have to modify a migration already executed on the rest of the CCNs to prevent unconsistencies on the database.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We are still using a constant (
DEFAULT_PRICE_AGGREGATE) with the same hardcoded default pricing than before. The only change is a type conversion fromDict[str, dict]toDict[ProductPriceType, ProductPricing]which is the type that the migration usesThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Anyway, I think that we don't have to touch an already executed migration, just to be sure that doesn't crash and for keeping the same migration history on all the CCNs.