|
1 | | -"""Joblib is a set of tools to provide **lightweight pipelining in |
2 | | -Python**. In particular: |
| 1 | +# Import necessary to preserve backward compatibility of pickles |
| 2 | +import sys |
| 3 | +import warnings |
3 | 4 |
|
4 | | -1. transparent disk-caching of functions and lazy re-evaluation |
5 | | - (memoize pattern) |
| 5 | +from joblib import * |
6 | 6 |
|
7 | | -2. easy simple parallel computing |
8 | 7 |
|
9 | | -Joblib is optimized to be **fast** and **robust** in particular on large |
10 | | -data and has specific optimizations for `numpy` arrays. It is |
11 | | -**BSD-licensed**. |
| 8 | +msg = ("sklearn.externals.joblib is deprecated in 0.21 and will be removed " |
| 9 | + "in 0.23. Please import this functionality directly from joblib, " |
| 10 | + "which can be installed with: pip install joblib. If this warning is " |
| 11 | + "raised when loading pickled models, you may need to re-serialize " |
| 12 | + "those models with scikit-learn 0.21+.") |
12 | 13 |
|
13 | | -
|
14 | | - ==================== =============================================== |
15 | | - **Documentation:** https://joblib.readthedocs.io |
16 | | -
|
17 | | - **Download:** http://pypi.python.org/pypi/joblib#downloads |
18 | | -
|
19 | | - **Source code:** http://github.com/joblib/joblib |
20 | | -
|
21 | | - **Report issues:** http://github.com/joblib/joblib/issues |
22 | | - ==================== =============================================== |
23 | | -
|
24 | | -
|
25 | | -Vision |
26 | | --------- |
27 | | -
|
28 | | -The vision is to provide tools to easily achieve better performance and |
29 | | -reproducibility when working with long running jobs. |
30 | | -
|
31 | | - * **Avoid computing twice the same thing**: code is rerun over an |
32 | | - over, for instance when prototyping computational-heavy jobs (as in |
33 | | - scientific development), but hand-crafted solution to alleviate this |
34 | | - issue is error-prone and often leads to unreproducible results |
35 | | -
|
36 | | - * **Persist to disk transparently**: persisting in an efficient way |
37 | | - arbitrary objects containing large data is hard. Using |
38 | | - joblib's caching mechanism avoids hand-written persistence and |
39 | | - implicitly links the file on disk to the execution context of |
40 | | - the original Python object. As a result, joblib's persistence is |
41 | | - good for resuming an application status or computational job, eg |
42 | | - after a crash. |
43 | | -
|
44 | | -Joblib addresses these problems while **leaving your code and your flow |
45 | | -control as unmodified as possible** (no framework, no new paradigms). |
46 | | -
|
47 | | -Main features |
48 | | ------------------- |
49 | | -
|
50 | | -1) **Transparent and fast disk-caching of output value:** a memoize or |
51 | | - make-like functionality for Python functions that works well for |
52 | | - arbitrary Python objects, including very large numpy arrays. Separate |
53 | | - persistence and flow-execution logic from domain logic or algorithmic |
54 | | - code by writing the operations as a set of steps with well-defined |
55 | | - inputs and outputs: Python functions. Joblib can save their |
56 | | - computation to disk and rerun it only if necessary:: |
57 | | -
|
58 | | - >>> from sklearn.externals.joblib import Memory |
59 | | - >>> cachedir = 'your_cache_dir_goes_here' |
60 | | - >>> mem = Memory(cachedir) |
61 | | - >>> import numpy as np |
62 | | - >>> a = np.vander(np.arange(3)).astype(np.float) |
63 | | - >>> square = mem.cache(np.square) |
64 | | - >>> b = square(a) # doctest: +ELLIPSIS |
65 | | - ________________________________________________________________________________ |
66 | | - [Memory] Calling square... |
67 | | - square(array([[0., 0., 1.], |
68 | | - [1., 1., 1.], |
69 | | - [4., 2., 1.]])) |
70 | | - ___________________________________________________________square - 0...s, 0.0min |
71 | | -
|
72 | | - >>> c = square(a) |
73 | | - >>> # The above call did not trigger an evaluation |
74 | | -
|
75 | | -2) **Embarrassingly parallel helper:** to make it easy to write readable |
76 | | - parallel code and debug it quickly:: |
77 | | -
|
78 | | - >>> from sklearn.externals.joblib import Parallel, delayed |
79 | | - >>> from math import sqrt |
80 | | - >>> Parallel(n_jobs=1)(delayed(sqrt)(i**2) for i in range(10)) |
81 | | - [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0] |
82 | | -
|
83 | | -
|
84 | | -3) **Fast compressed Persistence**: a replacement for pickle to work |
85 | | - efficiently on Python objects containing large data ( |
86 | | - *joblib.dump* & *joblib.load* ). |
87 | | -
|
88 | | -.. |
89 | | - >>> import shutil ; shutil.rmtree(cachedir) |
90 | | -
|
91 | | -""" |
92 | | - |
93 | | -# PEP0440 compatible formatted version, see: |
94 | | -# https://www.python.org/dev/peps/pep-0440/ |
95 | | -# |
96 | | -# Generic release markers: |
97 | | -# X.Y |
98 | | -# X.Y.Z # For bugfix releases |
99 | | -# |
100 | | -# Admissible pre-release markers: |
101 | | -# X.YaN # Alpha release |
102 | | -# X.YbN # Beta release |
103 | | -# X.YrcN # Release Candidate |
104 | | -# X.Y # Final release |
105 | | -# |
106 | | -# Dev branch marker is: 'X.Y.dev' or 'X.Y.devN' where N is an integer. |
107 | | -# 'X.Y.dev0' is the canonical version of 'X.Y.dev' |
108 | | -# |
109 | | -__version__ = '0.13.0' |
110 | | - |
111 | | - |
112 | | -from .memory import Memory, MemorizedResult, register_store_backend |
113 | | -from .logger import PrintTime |
114 | | -from .logger import Logger |
115 | | -from .hashing import hash |
116 | | -from .numpy_pickle import dump |
117 | | -from .numpy_pickle import load |
118 | | -from .compressor import register_compressor |
119 | | -from .parallel import Parallel |
120 | | -from .parallel import delayed |
121 | | -from .parallel import cpu_count |
122 | | -from .parallel import register_parallel_backend |
123 | | -from .parallel import parallel_backend |
124 | | -from .parallel import effective_n_jobs |
125 | | - |
126 | | -from .externals.loky import wrap_non_picklable_objects |
127 | | - |
128 | | - |
129 | | -__all__ = ['Memory', 'MemorizedResult', 'PrintTime', 'Logger', 'hash', 'dump', |
130 | | - 'load', 'Parallel', 'delayed', 'cpu_count', 'effective_n_jobs', |
131 | | - 'register_parallel_backend', 'parallel_backend', |
132 | | - 'register_store_backend', 'register_compressor', |
133 | | - 'wrap_non_picklable_objects'] |
| 14 | +if not hasattr(sys, "_is_pytest_session"): |
| 15 | + warnings.warn(msg, category=DeprecationWarning) |
0 commit comments