[sql-33] session: add migration code from kvdb to SQL #1051

ViktorTigerstrom · 2025-04-29T17:47:35Z

This PR introduces the migration logic for transitioning the sessions store from kvdb to SQL.

Note that as of this PR, the migration is not yet triggered by any production code, i.e. only tests execute the migration logic.

Part of #917

ViktorTigerstrom

Adding some review comments where input would be extra helpful :)!

ViktorTigerstrom · 2025-05-02T09:14:00Z

session/sql_store.go

+		macRecipe = &MacaroonRecipe{}
+		if perms != nil {
+			macRecipe.Permissions = unmarshalMacPerms(perms)
+		}
+		if caveats != nil {
+			macRecipe.Caveats = unmarshalMacCaveats(caveats)


Let me know if you think we should rather fix the kvdb store implementation rather than the sql one. But as we've previously opted for not updating the store behaviour, and make the sql one mimic the behaviour until we've completed the migration, I opted for this version here.

Alternatively, I could also override this in newly added overrideMacaroonRecipe function before the migration deep equals check. But I kept this as is, as I do think it's correct behaviour that we don't set the Permissions or Caveats and keep them at nil, if no value actually exists for them.

i think in this case it is fine to fix on the kvdb side so that our sql side can stay neat?

also commit message says fix kvdb&sql but this only touches sql

Please see #1051 (comment)

ViktorTigerstrom · 2025-05-02T09:17:17Z

session/sql_migration.go

+	if !session.RevokedAt.IsZero() {
+		err = tx.SetSessionRevokedAt(
+			ctx,
+			makeSetRevokedAtParams(sqlId, session.RevokedAt),
+		)
+		if err != nil {
+			return err
+		}
+	}


Could potentially update the insert session sql query to also take an optional RevokedAt sql.NullTime param, to not have to do this as a separate step. I opted for this version to keep things as is as much as possible, but if you think that'd be better, I'll update to do that instead :)

ViktorTigerstrom · 2025-05-02T09:20:45Z

session/sql_migration_test.go

+		{
+			"randomized sessions",
+			randomizedSessions,
+		},


Opted for a test that uses the "standard" rand lib here, as @bitromortac had a preference for that in #1047 (comment). But can also look into adding a test that uses rapid if you'd prefer.

ViktorTigerstrom · 2025-05-02T09:24:08Z

session/sql_migration_test.go

+			// We always have a caveat.Id, but the rest are
+			// randomized if they exist or not.
+			macCaveat.Id = randomBytes(rand.Intn(10) + 1)


Let me know if this is not true in for real world data, as migrations WILL fail if we don't set a caveat.Id when the caveats param of WithMacaroonRecipe is not nil.

Could potentially override this as well in the new overrideMacaroonRecipe as well. But I do not think this scenario can happen for real world data. But open to implementing that as well if you think it's worth doing it just to be sure.

ViktorTigerstrom · 2025-05-02T09:25:33Z

session/sql_migration_test.go

+// "dest" state, without checking if the state transition is legal.
+//
+// NOTE: this function should only be used for testing purposes.
+func (db *BoltStore) shiftStateUnsafe(_ context.Context, id ID,


Not fully sure if we add test functions like this one that's a struct member to the _test file, or if we add it in the actual struct file?

right, not too great, you could change to func shiftStateUnsafe(_ context.Context, db *BoltStore, id ID, dest State) error {

bitromortac

Great work, very clean on first pass! 🔥

bitromortac · 2025-05-07T13:16:17Z

session/sql_migration.go

+	// The remote public key is currently only set for autopilot sessions,
+	// else it's an empty byte array.


nit: I think it should read else it's a nil byte array, since we compare to nil in the code in other sites, which is also correctly set above with the var statement

session/sql_migration_test.go

bitromortac · 2025-05-07T13:55:11Z

session/sql_migration_test.go

+			require.NoError(t, err)
+
+			t.Cleanup(func() {
+				require.NoError(t, kvStore.DB.Close())


tiny nit: the embedded DB can be removed

bitromortac · 2025-05-07T13:57:31Z

session/sql_migration_test.go

+		// numberOfSessions is set to 100 to add enough sessions to get
+		// enough variation between number of invoices and payments, but
+		// kept low enough for the test not take too long to run, as the
+		// test time increases drastically by the number of sessions we
+		// migrate.


the doc mentions invoice and payments, which is probably a copy/paste

bitromortac · 2025-05-07T14:13:11Z

session/sql_migration_test.go

+// "dest" state, without checking if the state transition is legal.
+//
+// NOTE: this function should only be used for testing purposes.
+func (db *BoltStore) shiftStateUnsafe(_ context.Context, id ID,


right, not too great, you could change to func shiftStateUnsafe(_ context.Context, db *BoltStore, id ID, dest State) error {

ellemouton

holding off on review here until the prep PR discussed in the accounts mig PR is in (since that will be used to test this one)

ViktorTigerstrom · 2025-05-19T19:32:10Z

Now that #1047 has been merged, I'll update this PR to adhere to the feedback brought up in that PR, and then re-request reviews afterwards.

lightninglabs-deploy · 2025-05-26T20:21:40Z

@ViktorTigerstrom, remember to re-request review from reviewers when ready

ViktorTigerstrom · 2025-05-27T11:04:22Z

Updated the PR to also address feedback raised in #1047! Will address the small linting issue after next review round :)

bitromortac

LGTM, utACK 🎉

session/sql_migration.go

session/sql_migration_test.go

ViktorTigerstrom

Thanks a lot for the review @bitromortac 🎉!! I noticed that we can have real world data that sets a non-nil macaroon recipe, but with nil perms& caveats. I updated the code to be able to handle that case as well.

ViktorTigerstrom · 2025-05-28T11:49:50Z

session/sql_migration_test.go

+			},
+		},
+		{
+			name: "macaroon recipe with nil perms and caveats",


Note that this test is new, as this can actually happen for real world sessions.

ViktorTigerstrom · 2025-05-28T11:52:44Z

session/sql_store.go

+		macRecipe = &MacaroonRecipe{}
+		if perms != nil {
+			macRecipe.Permissions = unmarshalMacPerms(perms)
+		}
+		if caveats != nil {
+			macRecipe.Caveats = unmarshalMacCaveats(caveats)


Alternatively, I could also override this in newly added overrideMacaroonRecipe function before the migration deep equals check. But I kept this as is, as I do think it's correct behaviour that we don't set the Permissions or Caveats and keep them at nil, if no value actually exists for them.

ViktorTigerstrom · 2025-05-28T11:54:43Z

session/sql_migration_test.go

+			// We always have a caveat.Id, but the rest are
+			// randomized if they exist or not.
+			macCaveat.Id = randomBytes(rand.Intn(10) + 1)


Could potentially override this as well in the new overrideMacaroonRecipe as well. But I do not think this scenario can happen for real world data. But open to implementing that as well if you think it's worth doing it just to be sure.

ellemouton

looking good!

ellemouton · 2025-05-28T13:10:17Z

session/sql_store.go

+		macRecipe = &MacaroonRecipe{}
+		if perms != nil {
+			macRecipe.Permissions = unmarshalMacPerms(perms)
+		}
+		if caveats != nil {
+			macRecipe.Caveats = unmarshalMacCaveats(caveats)


i think in this case it is fine to fix on the kvdb side so that our sql side can stay neat?

ellemouton · 2025-05-28T13:10:37Z

session/sql_store.go

+		macRecipe = &MacaroonRecipe{}
+		if perms != nil {
+			macRecipe.Permissions = unmarshalMacPerms(perms)
+		}
+		if caveats != nil {
+			macRecipe.Caveats = unmarshalMacCaveats(caveats)


also commit message says fix kvdb&sql but this only touches sql

ellemouton · 2025-05-28T13:12:40Z

session/sql_migration.go

+	// If sessions are linked to a group, we must insert the initial session
+	// of each group before the other sessions in that group. This ensures
+	// we can retrieve the SQL group ID when inserting the remaining
+	// sessions. Therefore, we first insert all initial group sessions,


interesting - are we not able to rely on the fact that sessions are stored in order in kvdb? ie, if we get a session with a group ID set, that will always either point to the session at hand or a session we've already handled?

ie, i think we can just:

for s := sessions { if s.groupID = s.ID { <insert new session. no coupled group> continue } groupID := db.GetGroupIDByAlias(s.groupID) <insert new session. couple to group> }

or did you actually run into a scenario where these were not in order?

Hmmm so the listSessions for the kvdb returns the sessions sorted by the CreatedAt field. So given that this field actually contains timestamps in the correct order, your suggestion should work. But I see 2 potential scenarios where that wouldn't be true:

The user has managed to change the used time, so that linked sessions gets a CreatedAt timestamp prior to the initial session they are linking to.

Even though it's a real edge case, I believe that when we changed to using clock.Clock for litd, that we realised that this could have the side effect that the new timestamps would potentially get changed by +- a few hours unless I'm remembering this incorrectly? In that scenario, for some edge case users, they could have created the initial session before updating to the new litd version that added the clock.Clock implementation, and then linked a session to it right after which could then have an earlier CreatedAt timestamp than the initial session.

As I don't see any real cons with my current approach, other than it adding some more code complexity, I think it's a worth tradeoff than risking the migrations failing for such edge cases. Do you agree?

Hmmm so the listSessions for the kvdb returns the sessions sorted by the CreatedAt field

I'd opt for not using public methods like ListAllSessions for the migration. Rather let the migration manually get to the buckets/sub-buckets that it needs. We want to be able to change how ListAllSessions works without affecting the migration.

notice that the LND invoice migration does the same: takes a raw kvdb.Backend and *sqlc.Queries

im also more in favour of doing it this way cause then the IDs that the sessions have will be more intuitive as then (as long as the user didnt do anything strange) , the created at times will mostly be increasing at the IDs increase

Cool thanks! I updated this PR to instead pass a *bbolt.DB and directly iterate over the buckets, rather than using the public functions of the BoltStore. I've done the same for the accounts migration as well now with:
#1084

I've also looked through the previous migrations of the sessions kvdb as discussed offline, and none of them should be able to effect the order of the sessions entires in the kvdb store. However as also discussed offline, I didn't update the logic in the migrations, so that non-linked sessions are still migrated prior to linked sessions. I'm open to changing this though if you believe it's worth updating.

However as also discussed offline, I didn't update the logic in the migrations, so that non-linked sessions are still migrated prior to linked sessions. I'm open to changing this though if you believe it's worth updating.

ah, i thought the offline conclusion was the opposite 😅 cause i think it is just nice to have that the order in the DB is (at least on a best effort basis) the same as the order of creation.
Not completely blocking but i'd defs prefer it.

Sorry I should have been aware of this earlier! Since we insert the sessions in the kvdb store like this into the bucket:

if err := SerializeSession(&buf, session); err != nil { return err } return bucket.Put(getSessionKey(session), buf.Bytes())

Where the bucket is the bucket for the sessionBucketKey and the result of the getSessionKey will be session.LocalPublicKey.SerializeCompressed(), the order the sessions when iterating over the sessionBucketKey will actually be the lexicographical order of the serialized LocalPublicKey, and not in insertion order of the sessions.

Therefore there's no way to iterate over them in "insertion order", unless you know some way I'm unaware of :)? The closest we can get is to do what the kvdb store listSessions function does, which sorts the sessions by the CreatedAt timestamp, but that has the issue of what I brought up earlier with: #1051 (comment)

Therefore I think our best option that doesn't over complicate the code is to keep the migration code as what I've implemented today, i.e. first insert non-linked sessions, followed by inserting linked sessions?

ellemouton · 2025-05-28T13:20:26Z

session/sql_migration.go

+	session.AccountID.WhenSome(func(alias accounts.AccountID) {
+		// Check that the account exists in the SQL store, before


technically what we are doing here is just getting the account DB level ID.
we dont actually need to "check before link" since SQL does that for us.

ellemouton · 2025-05-28T13:20:45Z

session/sql_migration.go

+			return
+		}
+
+		acctID = sql.NullInt64{


sqldb.SQLInt64(accDBID)

Thanks, addressed these and others 🙏!

session/sql_migration.go

ellemouton · 2025-05-28T13:27:12Z

session/sql_migration.go

+		privacyFlagParams := sqlc.InsertSessionPrivacyFlagParams{
+			SessionID: sqlId,
+			Flag:      int32(privacyFlag),
+		}
+
+		err = tx.InsertSessionPrivacyFlag(ctx, privacyFlagParams)
+		if err != nil {
+			return err


this and many of the above can just be in-lined instead of having separate param vars

Addressed this and others :)

ViktorTigerstrom · 2025-06-03T01:18:09Z

I added a commit 6b8c58f at the end of the PR to address the feedback of #1051 (comment), as well as a fixup commit I will apply if you think the approach of added commit is good @ellemouton. Would therefore appreciate feedback on if this approach is what you had in mind when you suggested that I should change the kvdb implementation? Note that this actually changes how the MacaroonRecipe is persisted in kvdb, and it will set the field to null (which will also be represented when listing the session with litcli).

Note that I will fix the failing unit test tomorrow, that's due to my update of how kvdb implementation stores the MacaroonRecipe.

ViktorTigerstrom · 2025-06-03T10:27:25Z

Updated the PR by ensuring that the overrideMacaroonRecipe by itself handles the differences in how the MacaroonRecipe is represented between the kvdb & the SQL store. Therefore no changes are needed for how we'd persist it for neither the kvdb nor the sql stores.

@ellemouton, please let me know if you prefer that approach, or the other approach I've also implemented that changes how the kvdb persists the MacaroonRecipe.
Personally I'm in strong favour of letting the overrideMacaroonRecipe func handle the differences, and i.e. drop the last 2 commits that currently exists in this PR.

ellemouton

shweet! a few more comments.

Agreed with you that the override macaroon recipe approach is good 👍 i think i didnt understand correctly before that for bbolt we could still have the TLV actually be written with out any contents (right?) so yes, the overwrite makes sense rather than the "un-set"

ellemouton · 2025-06-04T11:51:42Z

session/interface.go

+	// Close closes the underlying store.
+	Close() error
+


i think you can drop this commit.

The interface should not expose details about the underlying store if possible. This exposes that the underlying store is something that needs to be closed. It also gives things we pass the interface to access to this Close method which we dont want.

ah, i think you're doing it cause of test cleanup - rather just call t.Cleanup sooner?

Cool thanks for the idea 🙏! Updated with a commit that calls t.Cleanup sooner instead.

ellemouton · 2025-06-04T11:53:39Z

session/sql_migration.go

+	// If sessions are linked to a group, we must insert the initial session
+	// of each group before the other sessions in that group. This ensures
+	// we can retrieve the SQL group ID when inserting the remaining
+	// sessions. Therefore, we first insert all initial group sessions,


Hmmm so the listSessions for the kvdb returns the sessions sorted by the CreatedAt field

I'd opt for not using public methods like ListAllSessions for the migration. Rather let the migration manually get to the buckets/sub-buckets that it needs. We want to be able to change how ListAllSessions works without affecting the migration.

ellemouton · 2025-06-04T11:54:35Z

session/sql_migration.go

+	// If sessions are linked to a group, we must insert the initial session
+	// of each group before the other sessions in that group. This ensures
+	// we can retrieve the SQL group ID when inserting the remaining
+	// sessions. Therefore, we first insert all initial group sessions,


it adding some more code complexity

yeah i think it's simpler and we have to worry about fewer things if we just directly iterate over the buckets.

ellemouton · 2025-06-04T11:55:36Z

session/sql_migration.go

+	// If sessions are linked to a group, we must insert the initial session
+	// of each group before the other sessions in that group. This ensures
+	// we can retrieve the SQL group ID when inserting the remaining
+	// sessions. Therefore, we first insert all initial group sessions,


now looking back, i think we should also change the accounts migration to not use the public Accounts method. Rather iterate directly over buckets.

ellemouton · 2025-06-04T11:56:49Z

session/sql_migration.go

+	// If sessions are linked to a group, we must insert the initial session
+	// of each group before the other sessions in that group. This ensures
+	// we can retrieve the SQL group ID when inserting the remaining
+	// sessions. Therefore, we first insert all initial group sessions,


in other words, rather than passing in a BoltStore, just pass in a *bbolt.DB instead (ie, the equivalent to SQLQueries)

ellemouton · 2025-06-04T11:57:27Z

session/sql_migration.go

+	// If sessions are linked to a group, we must insert the initial session
+	// of each group before the other sessions in that group. This ensures
+	// we can retrieve the SQL group ID when inserting the remaining
+	// sessions. Therefore, we first insert all initial group sessions,


notice that the LND invoice migration does the same: takes a raw kvdb.Backend and *sqlc.Queries

session/sql_migration.go

ellemouton · 2025-06-04T12:11:33Z

session/sql_migration.go

+	// If sessions are linked to a group, we must insert the initial session
+	// of each group before the other sessions in that group. This ensures
+	// we can retrieve the SQL group ID when inserting the remaining
+	// sessions. Therefore, we first insert all initial group sessions,


im also more in favour of doing it this way cause then the IDs that the sessions have will be more intuitive as then (as long as the user didnt do anything strange) , the created at times will mostly be increasing at the IDs increase

ViktorTigerstrom · 2025-06-09T12:29:38Z

Thanks for the latest review @ellemouton 🎉! Addressed your feedback with the latest push.

Added a new PR to address #1051 (comment), and rebased this PR on that though. This PR should be reviewable (but not mergeable) prior to that PR though, so just skip reviewing the first two commits when reviewing this.

ViktorTigerstrom · 2025-06-09T12:30:40Z

Also, re-requested a review from @bitromortac, as quite much has changed since your last review. Hope that's ok!

ellemouton

lgtm!
great work

ellemouton · 2025-06-10T07:31:24Z

accounts/sql_migration_test.go

@@ -39,9 +39,6 @@ func TestAccountStoreMigration(t *testing.T) {
 		*db.TransactionExecutor[SQLQueries]) {


fwiw for future, I dont think these 2 account commits needed to be in this PR as i dont think there is a dependency here. These can be reviewed and merged in parallel in whichever order

Did it that way so this wouldn't get merged without closing the stores in the tests, but I agree that it wouldn't be strictly necessary for merging :)

ellemouton · 2025-06-10T07:37:28Z

session/sql_migration.go

+	// If sessions are linked to a group, we must insert the initial session
+	// of each group before the other sessions in that group. This ensures
+	// we can retrieve the SQL group ID when inserting the remaining
+	// sessions. Therefore, we first insert all initial group sessions,


However as also discussed offline, I didn't update the logic in the migrations, so that non-linked sessions are still migrated prior to linked sessions. I'm open to changing this though if you believe it's worth updating.

ah, i thought the offline conclusion was the opposite 😅 cause i think it is just nice to have that the order in the DB is (at least on a best effort basis) the same as the order of creation.
Not completely blocking but i'd defs prefer it.

bitromortac

Nice, LGTM 🎉, just a question and a nit.

session/sql_migration.go

session/sql_migration_test.go

In preparation for upcoming migration tests from a kvdb to an SQL store, this commit updates the NewTestDB function to return the Store interface rather than a concrete store implementation. This change ensures that migration tests can call NewTestDB under any build tag while receiving a consistent return type.

We add a helper function to the functions that creates the test SQL stores, in order to ensure that the store is properly closed when the test is cleaned up.

This commit introduces the migration logic for transitioning the sessions store from kvdb to SQL. Note that as of this commit, the migration is not yet triggered by any production code, i.e. only tests execute the migration logic.

ViktorTigerstrom · 2025-06-12T11:26:42Z

Thanks for the reviews 🎉⚡!

ViktorTigerstrom added the no-changelog This PR is does not require a release notes entry label Apr 29, 2025

ViktorTigerstrom force-pushed the 2025-04-migrate-sessions branch from 449a2cc to 1828ec4 Compare April 29, 2025 17:52

levmi assigned ViktorTigerstrom May 1, 2025

ViktorTigerstrom force-pushed the 2025-04-migrate-sessions branch from 1828ec4 to a868a93 Compare May 2, 2025 09:04

ViktorTigerstrom commented May 2, 2025

View reviewed changes

ViktorTigerstrom requested review from ellemouton and bitromortac May 6, 2025 09:56

bitromortac reviewed May 7, 2025

View reviewed changes

ellemouton reviewed May 8, 2025

View reviewed changes

ViktorTigerstrom force-pushed the 2025-04-migrate-sessions branch from a868a93 to 4b3b39f Compare May 27, 2025 10:09

ViktorTigerstrom requested review from ellemouton and bitromortac May 27, 2025 11:04

bitromortac approved these changes May 27, 2025

View reviewed changes

session/sql_migration.go Show resolved Hide resolved

session/sql_migration_test.go Outdated Show resolved Hide resolved

ViktorTigerstrom mentioned this pull request May 27, 2025

[sql-41] firewalldb: add migration code for kvstores from kvdb to SQL #1079

Open

ViktorTigerstrom force-pushed the 2025-04-migrate-sessions branch 2 times, most recently from a48e9b2 to 8c970bf Compare May 28, 2025 11:46

ViktorTigerstrom commented May 28, 2025

View reviewed changes

ViktorTigerstrom force-pushed the 2025-04-migrate-sessions branch from 8c970bf to 46873cb Compare May 28, 2025 12:45

ellemouton reviewed May 28, 2025

View reviewed changes

ViktorTigerstrom force-pushed the 2025-04-migrate-sessions branch from 46873cb to 56f52c7 Compare June 3, 2025 00:56

ViktorTigerstrom requested a review from ellemouton June 3, 2025 01:18

ViktorTigerstrom force-pushed the 2025-04-migrate-sessions branch 2 times, most recently from c7901f0 to eabf4de Compare June 3, 2025 10:20

ellemouton reviewed Jun 4, 2025

View reviewed changes

ViktorTigerstrom mentioned this pull request Jun 9, 2025

[sql-42] accounts: Update SQL migration to iterate over buckets + ensure proper closure of SQL test stores #1084

Merged

ViktorTigerstrom force-pushed the 2025-04-migrate-sessions branch from eabf4de to 313cce1 Compare June 9, 2025 11:28

ViktorTigerstrom requested review from ellemouton and bitromortac June 9, 2025 12:29

ellemouton approved these changes Jun 10, 2025

View reviewed changes

bitromortac approved these changes Jun 11, 2025

View reviewed changes

session/sql_migration.go Show resolved Hide resolved

session/sql_migration_test.go Outdated Show resolved Hide resolved

ViktorTigerstrom added 4 commits June 12, 2025 11:52

session: ensure that test SQL store is closed

6c4c6a4

We add a helper function to the functions that creates the test SQL stores, in order to ensure that the store is properly closed when the test is cleaned up.

session: add migration code from kvdb to SQL

8f1aa99

This commit introduces the migration logic for transitioning the sessions store from kvdb to SQL. Note that as of this commit, the migration is not yet triggered by any production code, i.e. only tests execute the migration logic.

session: add randomized session migration test

2007d53

ViktorTigerstrom force-pushed the 2025-04-migrate-sessions branch 2 times, most recently from cb67dca to 2007d53 Compare June 12, 2025 11:15

ViktorTigerstrom merged commit 2bc7e7f into lightninglabs:master Jun 12, 2025
22 checks passed

		// The remote public key is currently only set for autopilot sessions,
		// else it's an empty byte array.

		session.AccountID.WhenSome(func(alias accounts.AccountID) {
		// Check that the account exists in the SQL store, before

		@@ -39,9 +39,6 @@ func TestAccountStoreMigration(t *testing.T) {
		*db.TransactionExecutor[SQLQueries]) {

[sql-33] session: add migration code from kvdb to SQL #1051

[sql-33] session: add migration code from kvdb to SQL #1051

Conversation

ViktorTigerstrom commented Apr 29, 2025

Uh oh!

ViktorTigerstrom left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bitromortac left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ellemouton left a comment

Choose a reason for hiding this comment

Uh oh!

ViktorTigerstrom commented May 19, 2025

Uh oh!

lightninglabs-deploy commented May 26, 2025

Uh oh!

ViktorTigerstrom commented May 27, 2025

Uh oh!

bitromortac left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

ViktorTigerstrom left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ellemouton left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!