Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enterprise blobber #1456

Draft
wants to merge 103 commits into
base: sprint-1.17
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
103 commits
Select commit Hold shift + click to select a range
573c473
remove vt mt wm and challenge workers
Hitenjain14 Jun 27, 2024
b28ba79
Merge branch 'feat/ref-index' of https://github.com/0chain/blobber in…
Hitenjain14 Jun 29, 2024
4178fc3
change apply changes
Hitenjain14 Jul 1, 2024
995912d
support upload,delete and createdir for enterprise
Hitenjain14 Jul 1, 2024
b34a7e8
remove unused columns in ref table
Hitenjain14 Jul 10, 2024
d273658
remove hash field from ref
Hitenjain14 Jul 11, 2024
85ec94f
parent id
Hitenjain14 Jul 11, 2024
a04e758
mkdir and version marker
Hitenjain14 Jul 12, 2024
e012e66
add version markers table
Hitenjain14 Jul 15, 2024
23e7eb9
add repair in version marker
Hitenjain14 Jul 21, 2024
7e3540d
add prev size and add vm to rollback
Hitenjain14 Jul 21, 2024
7454b0b
fix sql migration and use pointer for parent id
Hitenjain14 Jul 22, 2024
c0cfba3
fix ref table
Hitenjain14 Jul 22, 2024
854fac7
fix allocation table
Hitenjain14 Jul 22, 2024
8ff1e38
add info log
Hitenjain14 Jul 22, 2024
58cee84
Merge remote-tracking branch 'origin/sprint-1.17' into feat/enterpris…
Hitenjain14 Jul 22, 2024
de07b61
fix upload validation
Hitenjain14 Jul 22, 2024
75c745c
remove fileMeta
Hitenjain14 Jul 22, 2024
f8de5a9
init hasher
Hitenjain14 Jul 22, 2024
868c861
add log for file path
Hitenjain14 Jul 22, 2024
1c129ec
add check for root path
Hitenjain14 Jul 22, 2024
1a2edaa
add ts
Hitenjain14 Jul 22, 2024
4c0a6ab
add check for root ref
Hitenjain14 Jul 22, 2024
ce2543b
fix version marker id
Hitenjain14 Jul 22, 2024
0a6fe41
add file meta hash for dir
Hitenjain14 Jul 22, 2024
67f5ee6
fix allocation update in commit
Hitenjain14 Jul 22, 2024
2e13ad9
fix file path hash in commit write
Hitenjain14 Jul 22, 2024
22017d3
get hasher in base changer
Hitenjain14 Jul 23, 2024
3a4c4df
check for created refs
Hitenjain14 Jul 23, 2024
dd6f54e
add check for dir in delete
Hitenjain14 Jul 23, 2024
a7d405f
fix get refs and add is empty and alloc version to ref
Hitenjain14 Jul 25, 2024
0321e66
fix build
Hitenjain14 Jul 25, 2024
b1985d1
fix seq cache and add logs
Hitenjain14 Jul 25, 2024
7d35775
use or
Hitenjain14 Jul 25, 2024
bc2cadc
add check for 0 id
Hitenjain14 Jul 25, 2024
991a7ed
fix parent ref id
Hitenjain14 Jul 25, 2024
58e4317
fix get parent paths
Hitenjain14 Jul 26, 2024
7719bd7
fix dir list fields
Hitenjain14 Jul 26, 2024
c62ddaa
fix dirlist tag for allocation version
Hitenjain14 Jul 26, 2024
79c0bf2
add check for parent path of root
Hitenjain14 Jul 26, 2024
12b4e25
add id to select for dir
Hitenjain14 Jul 26, 2024
1bce447
set model
Hitenjain14 Jul 26, 2024
50f6f1c
fix num block column
Hitenjain14 Jul 26, 2024
5a0559c
add check for return directory
Hitenjain14 Jul 26, 2024
20cfb0f
add a check for level in get refs
Hitenjain14 Jul 26, 2024
de124d0
order by level and path
Hitenjain14 Jul 26, 2024
c367f4e
rever order by level
Hitenjain14 Jul 26, 2024
2f90507
fix query
Hitenjain14 Jul 26, 2024
e1163d3
remove unused fields in select
Hitenjain14 Jul 26, 2024
8b6646f
add check for empty dir in apply change
Hitenjain14 Jul 26, 2024
a2e49b9
fix build and add check for update allocation
Hitenjain14 Jul 26, 2024
1bb9f00
add move,copy and rename
Hitenjain14 Jul 27, 2024
36fa635
add logs
Hitenjain14 Jul 27, 2024
84f875a
set name
Hitenjain14 Jul 27, 2024
c92a62a
add deleted at
Hitenjain14 Jul 27, 2024
4bcd3a4
add collector cache
Hitenjain14 Jul 27, 2024
a604d5d
fix index
Hitenjain14 Jul 27, 2024
0606901
init map
Hitenjain14 Jul 27, 2024
adb7fac
add new indexes
Hitenjain14 Jul 27, 2024
60cf8c8
update is_precommit index
Hitenjain14 Jul 27, 2024
8b91732
use materialized path
Hitenjain14 Jul 29, 2024
67bb51b
hardcode limit values
Hitenjain14 Jul 29, 2024
26733c6
remove parent path index
Hitenjain14 Jul 29, 2024
5050629
fix type
Hitenjain14 Jul 29, 2024
66d466f
remove ref type
Hitenjain14 Jul 29, 2024
071b9c5
revert index changes
Hitenjain14 Jul 29, 2024
ae7a84f
Merge branch 'feat/enterprise-blobber' of https://github.com/0chain/b…
Hitenjain14 Jul 29, 2024
5330ba8
Use alloc version in place of precommit (#1465)
Hitenjain14 Aug 9, 2024
f49dd1b
Merge remote-tracking branch 'origin' into feat/enterprise-blobber
Hitenjain14 Aug 9, 2024
5a25266
add size for root dir
Hitenjain14 Aug 10, 2024
bd9ccdb
Merge remote-tracking branch 'origin/sprint-1.17' into feat/enterpris…
Hitenjain14 Aug 14, 2024
d748809
rmv hash len check in get path for file
Hitenjain14 Aug 18, 2024
dd6787f
add slash seperator
Hitenjain14 Sep 6, 2024
350bb7e
fix list root
Hitenjain14 Sep 24, 2024
f8aefee
add log for connection obj
Hitenjain14 Sep 24, 2024
0cbbfe2
Merge branch 'feat/enterprise-blobber' of https://github.com/0chain/b…
Hitenjain14 Sep 24, 2024
ffb861c
add log for blacklist
Hitenjain14 Sep 27, 2024
dfd6a26
add get refs log
Hitenjain14 Sep 30, 2024
991b638
add return
Hitenjain14 Sep 30, 2024
37296f3
fix concurrent read and write
Hitenjain14 Oct 27, 2024
421e1a4
add unlock
Hitenjain14 Oct 27, 2024
66ca94e
fix duplicate dir (#1504)
Hitenjain14 Nov 11, 2024
6c1952f
update limit monthly
Hitenjain14 Dec 9, 2024
f69d989
remove temp size check
Hitenjain14 Dec 12, 2024
d2b4c26
Fix auth ticket enterprise
Jayashsatolia403 Jan 6, 2025
06d5ad9
Debug
Jayashsatolia403 Jan 6, 2025
0eae2e7
Merge pull request #1528 from 0chain/fix/auth-ticket-enterprise
dabasov Jan 11, 2025
68f7733
dummy commit
Hitenjain14 Jan 27, 2025
d46f49e
Set enterprise = true always
Jayashsatolia403 Jan 29, 2025
2d169e8
Revert "Set enterprise = true always"
Jayashsatolia403 Jan 31, 2025
e2dfba1
sv set to 1
smaulik13 Jan 31, 2025
3d08335
fix size and temp dir path
Hitenjain14 Feb 5, 2025
ebc3c09
dummy commit
smaulik13 Feb 6, 2025
746b208
logging
smaulik13 Feb 14, 2025
9af257f
log
smaulik13 Feb 14, 2025
8c8e45c
logging
smaulik13 Feb 20, 2025
b41ceaa
logging
smaulik13 Feb 20, 2025
77a6fcb
common sdk integrate
smaulik13 Feb 23, 2025
99a44a4
conflict fix
smaulik13 Feb 24, 2025
1234d58
Merge branch 'sprint-1.17' into feat/enterprise-blobber
smaulik13 Feb 24, 2025
9df7072
go mod update
smaulik13 Feb 24, 2025
1e12d3a
version fix
smaulik13 Feb 24, 2025
e54c557
added logs
smaulik13 Feb 26, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion code/go/0chain.net/blobber/logging.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import (

"github.com/0chain/blobber/code/go/0chain.net/blobbercore/config"
"github.com/0chain/blobber/code/go/0chain.net/core/logging"
"github.com/0chain/gosdk/zcncore"
"github.com/0chain/gosdk_common/zcncore"
)

func setupLogging() {
Expand Down
20 changes: 12 additions & 8 deletions code/go/0chain.net/blobber/settings.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import (

"github.com/0chain/blobber/code/go/0chain.net/blobbercore/config"
"github.com/0chain/blobber/code/go/0chain.net/core/logging"
"github.com/0chain/gosdk/zcncore"
"github.com/0chain/gosdk_common/core/transaction"
)

type storageScCB struct {
Expand Down Expand Up @@ -58,20 +58,24 @@ func (ssc *storageScCB) OnInfoAvailable(op int, status int, info string, errStr
}

func setStorageScConfigFromChain() error {
cb := &storageScCB{
done: make(chan struct{}),
conf, err := transaction.GetConfig("storage_sc_config")
if err != nil {
return err
}
err := zcncore.GetStorageSCConfig(cb)

maxChallengeCompletionRoundsString := conf.Fields["max_challenge_completion_rounds"]
maxChallengeCompletionRoundsInt, err := strconv.ParseInt(maxChallengeCompletionRoundsString, 10, 64)
if err != nil {
return err
}
<-cb.done
if cb.err != nil {
maxFileSizeString := conf.Fields["max_file_size"]
maxFileSizeInt64, err := strconv.ParseInt(maxFileSizeString, 10, 64)
if err != nil {
return err
}

config.StorageSCConfig.ChallengeCompletionTime = cb.cct
config.StorageSCConfig.MaxFileSize = cb.mfs
config.StorageSCConfig.ChallengeCompletionTime = maxChallengeCompletionRoundsInt
config.StorageSCConfig.MaxFileSize = maxFileSizeInt64
return nil
}

Expand Down
29 changes: 21 additions & 8 deletions code/go/0chain.net/blobber/zcn.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ package main

import (
"fmt"
"time"

"github.com/0chain/blobber/code/go/0chain.net/blobbercore/config"
"github.com/0chain/blobber/code/go/0chain.net/blobbercore/filestore"
"github.com/0chain/blobber/code/go/0chain.net/blobbercore/handler"
Expand All @@ -10,10 +12,9 @@ import (
handleCommon "github.com/0chain/blobber/code/go/0chain.net/core/common/handler"
"github.com/0chain/blobber/code/go/0chain.net/core/logging"
"github.com/0chain/blobber/code/go/0chain.net/core/node"
"github.com/0chain/gosdk/zboxcore/sdk"
"github.com/0chain/gosdk/zcncore"
"github.com/0chain/gosdk_common/core/client"
"github.com/0chain/gosdk_common/zcncore"
"go.uber.org/zap"
"time"
)

func registerOnChain() error {
Expand Down Expand Up @@ -82,16 +83,28 @@ func setupServerChain() error {
serverChain := chain.NewChainFromConfig()
chain.SetServerChain(serverChain)

if err := zcncore.InitZCNSDK(serverChain.BlockWorker, config.Configuration.SignatureScheme); err != nil {
//options := []int{
// 0,
// 10, // MinConfirmation
// 20, // MinSubmit
// 3, // ConfirmationChainLength
// 3, // SharderConsensous
// 1, // QuerySleepTime
// 0, // VerifyOptimistic
//}

err := client.InitSDK("{}", serverChain.BlockWorker, config.Configuration.ChainID, config.Configuration.SignatureScheme, 0, false)
if err != nil {
return err
}
if err := zcncore.SetWalletInfo(node.Self.GetWalletString(), false); err != nil {

err = zcncore.SetGeneralWalletInfo(node.Self.GetWalletString(), config.Configuration.SignatureScheme)
if err != nil {
return err
}

if err := sdk.InitStorageSDK(node.Self.GetWalletString(), serverChain.BlockWorker, config.Configuration.ChainID, config.Configuration.SignatureScheme,
nil, 0); err != nil {
return err
if node.Self.GetWallet().IsSplit {
zcncore.RegisterZauthServer(serverChain.ZauthServer)
}

fmt.Print(" [OK]\n")
Expand Down
Loading
Loading