Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build 1.6.7 version pip package #386

Open
wants to merge 55 commits into
base: buildpackage
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
f96aa4b
update README: clone submodule (#323)
suiguoxin Jul 28, 2022
a192a17
Update git clone command to include ThirdParty submodules (#326)
phgn0 Jul 29, 2022
465eb3b
improve performance and memory usage (#321)
MaggieQi Aug 10, 2022
759758c
add tools for opq training and inference (#324)
jinwei14 Aug 31, 2022
60f709b
Fix build errors for GPU PQ
bkarsin Sep 15, 2022
4d2eccf
Added more error checking for GPU PQ
bkarsin Sep 19, 2022
4cabd0b
Small fixes
bkarsin Sep 20, 2022
5ce6fd2
Fix Windows build error
bkarsin Sep 22, 2022
2b17965
Fix zstd build/link error
bkarsin Sep 22, 2022
77b93a2
Fix for Windows build
bkarsin Sep 22, 2022
5112fff
Add SIMDUtils.cpp to GPU build
Diego-Cai Sep 22, 2022
6242cff
Try to trigger Azure Pipeline CL build
Diego-Cai Sep 26, 2022
cbc4e0d
Remove space
Diego-Cai Sep 27, 2022
ca61760
Merge pull request #334 from bkarsin/fix_gpu_pq_build
caijiediego Sep 27, 2022
55ca655
Add GPU KNN to Generate Truth (#336)
caijiediego Oct 8, 2022
f7e3fe4
Fix low accuracy issue with GPU index build for int8/cosine configura…
bkarsin Nov 1, 2022
636b3d6
Add nni_auto_tune example (#325)
smallv0221 Nov 1, 2022
98ef3b1
Fix graph accuracy of GPU build for PQ/OPQ (#340)
bkarsin Nov 3, 2022
3d76e5e
Add 768D support for GPU Generate Ground Truth (#343)
caijiediego Nov 3, 2022
ebeb690
Fix windows build issues for GPU code (#346)
bkarsin Nov 17, 2022
2d2cfb3
fix bkt bug: shuffle bug when clustering (#349)
Yuming-Xu Nov 29, 2022
dda1180
Fix python version (#333)
MaggieQi Dec 2, 2022
de5b7f8
add logger for total distance (#351)
jinwei14 Jan 6, 2023
9c777df
Prevent saving failure in SaveindexToFile (#355)
PhilipBAdams Feb 21, 2023
a5bd48e
Fix static initialization fiasco problem with the logger and code cal…
asklar Feb 22, 2023
a9fb933
Add WinRT wrapper (#360)
asklar Mar 1, 2023
0207479
add nuget package for WinRT (#361)
asklar Mar 1, 2023
5ffeabe
Thread local context (#359)
MaggieQi Mar 9, 2023
d082b55
Update WindowsInstallation.md (#365)
MS-Renan Mar 13, 2023
b725cac
Update versions in requirements.txt to patched versions (#367)
0x4248 Mar 15, 2023
e950fad
add filter support for BKT index (#368)
zqxjjj Mar 16, 2023
fe03712
Rename logging macro to avoid name conflicts (#366)
PhilipBAdams Mar 16, 2023
ac85ff4
Allow setting a different Logger at runtime (#370)
PhilipBAdams Mar 21, 2023
72f929d
add filter support for BKT: replace function pointer to std::fucntion…
zqxjjj Mar 23, 2023
0341c33
Add protections against overflow and size mismatch, and invalid IDs i…
PhilipBAdams Mar 23, 2023
d6aca40
Add dimension check for ProcessWithoutMPI. (#373)
yangliu-ms Mar 27, 2023
2c0bb04
resolve compilation warnings (#374)
limenghao Apr 2, 2023
fc24d65
Fix 384d GPU index build support and error reporting (#363)
bkarsin Apr 2, 2023
3b00327
Clr core (#376)
MaggieQi Apr 4, 2023
0df1b75
std::out_of_range doesn't have a default constructor (#378)
PhilipBAdams Apr 10, 2023
98bc296
Remove old cuda tests and fix linux build error (#381)
bkarsin May 4, 2023
ccb78d6
enhance syncing script uploading and downloading (#382)
jinwei14 May 4, 2023
268578d
Avoid off-by-one in BKT search (#383)
PhilipBAdams May 9, 2023
b28eafa
Clr core (#385)
MaggieQi May 24, 2023
ff3825d
Microsoft mandatory file
microsoft-github-policy-service[bot] Jun 2, 2023
2ffaec6
Auto merge mandatory file pr
microsoft-github-policy-service[bot] Jun 12, 2023
c05bdc8
More gracefully handle when invalid IDs are inserted or checked in th…
PhilipBAdams Jul 18, 2023
35a9bd1
Convenience functions for parsing SPANN index (#384)
PhilipBAdams Aug 15, 2023
176e7cd
When storing metadata, we were truncating at the first '\n' which is …
asklar Aug 28, 2023
7e9ff64
Opt perf and fix thread affinity bug (#395)
MaggieQi Oct 11, 2023
1f6a85e
fix compile failed, ErrorCode::fail should be ErrorCode::Fail (#404)
dlmodric Dec 15, 2023
0dedbd9
Iterator interface (#407)
zqxjjj Feb 2, 2024
e7e4f71
add Wait() method to SPTAG::Helper::DiskIO (#409)
hliu18 Apr 27, 2024
9d7da69
Bump pillow from 9.4.0 to 10.0.1 in /docs/examples (#399)
dependabot[bot] May 1, 2024
4ecf249
[Draft] GPU unit and integration tests (#377)
bkarsin May 3, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix static initialization fiasco problem with the logger and code cal…
…ling into it from other TUs (#358)

* Addresses static initialization fiasco problem with the logger and code calling into it from other translation units

* fix namespace

* make logger init multithreading safe

* actually we can use magic statics to simplify

* Fix #356

* gate GMH/GPA on Windows only
asklar authored Feb 22, 2023
commit a5bd48e8075e72ea679db6ef0388cb5744f06cfe
4 changes: 2 additions & 2 deletions AnnService/inc/Core/Common.h
Original file line number Diff line number Diff line change
@@ -123,9 +123,9 @@ extern std::shared_ptr<Helper::DiskIO>(*f_createIO)();
#define IOBINARY(ptr, func, bytes, ...) if (ptr->func(bytes, __VA_ARGS__) != bytes) return ErrorCode::DiskIOFail
#define IOSTRING(ptr, func, ...) if (ptr->func(__VA_ARGS__) == 0) return ErrorCode::DiskIOFail

extern std::shared_ptr<Helper::Logger> g_pLogger;
extern std::shared_ptr<Helper::Logger> GetLogger();

#define LOG(l, ...) g_pLogger->Logging("SPTAG", l, __FILE__, __LINE__, __FUNCTION__, __VA_ARGS__)
#define LOG(l, ...) GetLogger()->Logging("SPTAG", l, __FILE__, __LINE__, __FUNCTION__, __VA_ARGS__)

class MyException : public std::exception
{
2 changes: 1 addition & 1 deletion AnnService/inc/SSDServing/SSDIndex.h
Original file line number Diff line number Diff line change
@@ -174,7 +174,7 @@ namespace SPTAG {

if (!p_opts.m_logFile.empty())
{
g_pLogger.reset(new Helper::FileLogger(Helper::LogLevel::LL_Info, p_opts.m_logFile.c_str()));
GetLogger().reset(new Helper::FileLogger(Helper::LogLevel::LL_Info, p_opts.m_logFile.c_str()));
}
int numThreads = p_opts.m_iSSDNumberOfThreads;
int internalResultNum = p_opts.m_searchInternalResultNum;
16 changes: 14 additions & 2 deletions AnnService/src/Core/VectorIndex.cpp
Original file line number Diff line number Diff line change
@@ -15,11 +15,23 @@ typedef typename SPTAG::Helper::Concurrent::ConcurrentMap<std::string, SPTAG::Si

using namespace SPTAG;

std::shared_ptr<Helper::Logger> SPTAG::GetLogger() {
#ifdef DEBUG
std::shared_ptr<Helper::Logger> SPTAG::g_pLogger(new Helper::SimpleLogger(Helper::LogLevel::LL_Debug));
auto logLevel = Helper::LogLevel::LL_Debug;
#else
std::shared_ptr<Helper::Logger> SPTAG::g_pLogger(new Helper::SimpleLogger(Helper::LogLevel::LL_Info));
auto logLevel = Helper::LogLevel::LL_Info;
#endif
#ifdef _WINDOWS_
if (auto exeHandle = GetModuleHandleW(nullptr)) {
if (auto SPTAG_GetLoggerLevel = reinterpret_cast<SPTAG::Helper::LogLevel(*)()>(GetProcAddress(exeHandle, "SPTAG_GetLoggerLevel"))) {
logLevel = SPTAG_GetLoggerLevel();
}
}
#endif // _WINDOWS_

static std::shared_ptr<Helper::Logger> s_pLogger = std::make_shared<Helper::SimpleLogger>(logLevel);
return s_pLogger;
}

std::mt19937 SPTAG::rg;

2 changes: 1 addition & 1 deletion AnnService/src/Quantizer/main.cpp
Original file line number Diff line number Diff line change
@@ -68,7 +68,7 @@ void QuantizeAndSave(std::shared_ptr<SPTAG::Helper::VectorSetReader>& vectorRead

int main(int argc, char* argv[])
{
std::shared_ptr<QuantizerOptions> options = std::make_shared<QuantizerOptions>(10000, true, 0.0, SPTAG::QuantizerType::None, std::string(), -1, std::string(), std::string());
std::shared_ptr<QuantizerOptions> options = std::make_shared<QuantizerOptions>(10000, true, 0.0f, SPTAG::QuantizerType::None, std::string(), -1, std::string(), std::string());

if (!options->Parse(argc - 1, argv + 1))
{
2 changes: 1 addition & 1 deletion AnnService/src/Server/SearchService.cpp
Original file line number Diff line number Diff line change
@@ -84,7 +84,7 @@ SearchService::Initialize(int p_argNum, char* p_args[])
}

if (!cmdOptions.m_logFile.empty()) {
g_pLogger.reset(new Helper::FileLogger(Helper::LogLevel::LL_Debug, cmdOptions.m_logFile.c_str()));
GetLogger().reset(new Helper::FileLogger(Helper::LogLevel::LL_Debug, cmdOptions.m_logFile.c_str()));
}

m_serviceContext.reset(new ServiceContext(cmdOptions.m_configFile));