1
+ ** Note: Maintainers needed : Those that committed in the past code to this repo or are presenting new PRs and have experience and interest on helping to maintain repos & python libraries (code quality, testing, integration, etc). If you are interested on getting our PR's through and helping others to contribute to the library, please get in touch.**
1
2
2
- ===============
3
- CMRESHandler.py
4
- ===============
3
+ ----
5
4
6
- | |license| |versions| |status| |downloads|
7
- | |ci_status| |codecov| |gitter|
5
+ [ ![ downloads] ( https://img.shields.io/pypi/dd/CMRESHandler.svg )] ( https://pypi.python.org/pypi/CMRESHandler )
6
+ [ ![ versions] ( https://img.shields.io/pypi/pyversions/CMRESHandler.svg )] ( https://pypi.python.org/pypi/CMRESHandler )
7
+ [ ![ status] ( https://img.shields.io/pypi/status/CMRESHandler.svg )] ( https://pypi.python.org/pypi/CMRESHandler )
8
+ [ ![ license] ( https://img.shields.io/pypi/l/CMRESHandler.svg )] ( https://pypi.python.org/pypi/CMRESHandler )
9
+ [ ![ ci_status] ( https://travis-ci.org/cmanaha/python-elasticsearch-logger.svg?branch=master )] ( https://travis-ci.org/cmanaha/python-elasticsearch-logger )
10
+ [ ![ codecov] ( https://codecov.io/github/cmanaha/python-elasticsearch-logger/coverage.svg?branch=master )] ( http://codecov.io/github/cmanaha/python-elasticsearch-logger?branch=master )
11
+ [ ![ gitter] ( https://badges.gitter.im/Join%20Chat.svg )] ( https://gitter.im/cmanaha/python-elasticsearch-logger?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge )
8
12
9
-
10
- **Note: Maintainers needed : Those that committed in the past code to this repo or are presenting new PRs and have experience and interest on helping to maintain repos & python libraries (code quality, testing, integration, etc). If you are intereted on getting our PR's through and helping others to contribute to the library, please get in touch. **
11
-
12
-
13
- Python Elasticsearch Log handler
14
- ********************************
13
+ # Python Elasticsearch Log handler
15
14
16
15
This library provides an Elasticsearch logging appender compatible with the
17
16
python standard ` logging <https://docs.python.org/2/library/logging.html> ` _ library.
@@ -20,58 +19,62 @@ The code source is in github at `https://github.com/cmanaha/python-elasticsearch
20
19
< https://github.com/cmanaha/python-elasticsearch-logger > `_
21
20
22
21
23
- Installation
24
- ============
22
+ # Installation
23
+
25
24
Install using pip::
26
25
27
26
pip install CMRESHandler
28
27
29
- Requirements Python 2
30
- =====================
28
+ # Requirements Python 2
29
+
31
30
This library requires the following dependencies
32
31
- elasticsearch
33
32
- requests
34
33
- enum
35
34
36
35
37
- Requirements Python 3
38
- =====================
36
+ # Requirements Python 3
37
+
39
38
This library requires the following dependencies
40
39
- elasticsearch
41
40
- requests
42
41
43
- Additional requirements for Kerberos support
44
- ============================================
45
- Additionally, the package support optionally kerberos authentication by adding the following dependecy
42
+ # Additional requirements for Kerberos support
43
+
44
+ Additionally, the package support optionally kerberos authentication by adding the following dependency
46
45
- requests-kerberos
47
46
48
- Additional requirements for AWS IAM user authentication (request signing)
49
- =========================================================================
50
- Additionally, the package support optionally AWS IAM user authentication by adding the following dependecy
47
+ # Additional requirements for AWS IAM user authentication (request signing)
48
+
49
+ Additionally, the package support optionally AWS IAM user authentication by adding the following dependency
51
50
- requests-aws4auth
52
51
53
- Using the handler in your program
54
- ==================================
55
- To initialise and create the handler, just add the handler to your logger as follow ::
52
+ # Using the handler in your program
56
53
57
- from cmreslogging.handlers import CMRESHandler
58
- handler = CMRESHandler(hosts=[{'host': 'localhost', 'port': 9200}],
59
- auth_type=CMRESHandler.AuthType.NO_AUTH,
60
- es_index_name="my_python_index")
61
- log = logging.getLogger("PythonTest")
62
- log.setLevel(logging.INFO)
63
- log.addHandler(handler)
54
+ To initialize and create the handler, just add the handler to your logger as follow ::
55
+
56
+ ``` python
57
+ from cmreslogging.handlers import CMRESHandler
58
+ handler = CMRESHandler(hosts = [{' host' : ' localhost' , ' port' : 9200 }],
59
+ auth_type = CMRESHandler.AuthType.NO_AUTH ,
60
+ es_index_name = " my_python_index" )
61
+ log = logging.getLogger(" PythonTest" )
62
+ log.setLevel(logging.INFO )
63
+ log.addHandler(handler)
64
+ ```
64
65
65
66
You can add fields upon initialisation, providing more data of the execution context ::
66
67
67
- from cmreslogging.handlers import CMRESHandler
68
- handler = CMRESHandler(hosts=[{'host': 'localhost', 'port': 9200}],
69
- auth_type=CMRESHandler.AuthType.NO_AUTH,
70
- es_index_name="my_python_index",
71
- es_additional_fields={'App': 'MyAppName', 'Environment': 'Dev'})
72
- log = logging.getLogger("PythonTest")
73
- log.setLevel(logging.INFO)
74
- log.addHandler(handler)
68
+ ``` python
69
+ from cmreslogging.handlers import CMRESHandler
70
+ handler = CMRESHandler(hosts = [{' host' : ' localhost' , ' port' : 9200 }],
71
+ auth_type = CMRESHandler.AuthType.NO_AUTH ,
72
+ es_index_name = " my_python_index" ,
73
+ es_additional_fields = {' App' : ' MyAppName' , ' Environment' : ' Dev' })
74
+ log = logging.getLogger(" PythonTest" )
75
+ log.setLevel(logging.INFO )
76
+ log.addHandler(handler)
77
+ ```
75
78
76
79
This additional fields will be applied to all logging fields and recorded in elasticsearch
77
80
@@ -82,18 +85,20 @@ To log, use the regular commands from the logging library ::
82
85
Your code can also dump additional extra fields on a per log basis that can be used to instrument
83
86
operations. For example, when reading information from a database you could do something like::
84
87
85
- start_time = time.time()
86
- database_operation()
87
- db_delta = time.time() - start_time
88
- log.debug("DB operation took %.3f seconds" % db_delta, extra={'db_execution_time': db_delta})
88
+ ``` python
89
+ start_time = time.time()
90
+ database_operation()
91
+ db_delta = time.time() - start_time
92
+ log.debug(" DB operation took %.3f seconds" % db_delta, extra = {' db_execution_time' : db_delta})
93
+ ```
89
94
90
95
The code above executes the DB operation, measures the time it took and logs an entry that contains
91
96
in the message the time the operation took as string and for convenience, it creates another field
92
97
called db_execution_time with a float that can be used to plot the time this operations are taking using
93
98
Kibana on top of elasticsearch
94
99
95
- Initialisation parameters
96
- =========================
100
+ # Initialization parameters
101
+
97
102
The constructors takes the following parameters:
98
103
- hosts: The list of hosts that elasticsearch clients will connect, multiple hosts are allowed, for example ::
99
104
@@ -118,54 +123,56 @@ The constructors takes the following parameters:
118
123
- es_doc_type: A string with the name of the document type that will be used `` python_log `` used by default
119
124
- es_additional_fields: A dictionary with all the additional fields that you would like to add to the logs
120
125
121
- Django Integration
122
- ==================
126
+ # Django Integration
127
+
123
128
It is also very easy to integrate the handler to ` Django <https://www.djangoproject.com/> ` _ And what is even
124
129
better, at DEBUG level django logs information such as how long it takes for DB connections to return so
125
130
they can be plotted on Kibana, or the SQL statements that Django executed. ::
126
131
127
- from cmreslogging.handlers import CMRESHandler
128
- LOGGING = {
129
- 'version': 1,
130
- 'disable_existing_loggers ': False ,
131
- 'handlers ': {
132
- 'file ': {
133
- 'level ': 'DEBUG',
134
- 'class ': 'logging.handlers.RotatingFileHandler ',
135
- 'filename ': './debug.log ',
136
- 'maxBytes ': 102400 ,
137
- 'backupCount ': 5 ,
138
- } ,
139
- 'elasticsearch': {
140
- 'level ': 'DEBUG',
141
- 'class ': 'cmreslogging.handlers.CMRESHandler ',
142
- 'hosts ': [{'host': 'localhost', 'port': 9200}] ,
143
- 'es_index_name ': 'my_python_app' ,
144
- 'es_additional_fields ': {'App': 'Test', 'Environment': 'Dev'} ,
145
- 'auth_type ': CMRESHandler.AuthType.NO_AUTH ,
146
- 'use_ssl ': False ,
147
- } ,
132
+ ``` python
133
+ from cmreslogging.handlers import CMRESHandler
134
+ LOGGING = {
135
+ ' version ' : 1 ,
136
+ ' disable_existing_loggers ' : False ,
137
+ ' handlers ' : {
138
+ ' file ' : {
139
+ ' level ' : ' DEBUG ' ,
140
+ ' class ' : ' logging.handlers.RotatingFileHandler ' ,
141
+ ' filename ' : ' ./debug.log ' ,
142
+ ' maxBytes ' : 102400 ,
143
+ ' backupCount ' : 5 ,
144
+ },
145
+ ' elasticsearch ' : {
146
+ ' level ' : ' DEBUG ' ,
147
+ ' class ' : ' cmreslogging.handlers.CMRESHandler ' ,
148
+ ' hosts ' : [{ ' host ' : ' localhost ' , ' port ' : 9200 }] ,
149
+ ' es_index_name ' : ' my_python_app ' ,
150
+ ' es_additional_fields ' : { ' App ' : ' Test ' , ' Environment ' : ' Dev ' } ,
151
+ ' auth_type ' : CMRESHandler.AuthType. NO_AUTH ,
152
+ ' use_ssl ' : False ,
148
153
},
149
- 'loggers': {
150
- 'django ': {
151
- 'handlers ': ['file','elasticsearch'],
152
- 'level ': 'DEBUG' ,
153
- 'propagate ': True ,
154
- } ,
154
+ },
155
+ ' loggers ' : {
156
+ ' django ' : {
157
+ ' handlers ' : [ ' file ' , ' elasticsearch ' ] ,
158
+ ' level ' : ' DEBUG ' ,
159
+ ' propagate ' : True ,
155
160
},
156
- }
161
+ },
162
+ }
163
+ ```
157
164
158
165
There is more information about how Django logging works in the
159
166
` Django documentation <https://docs.djangoproject.com/en/1.9/topics/logging//> ` _
160
167
161
168
162
- Building the sources & Testing
163
- ------------------------------
169
+ ## Building the sources & Testing
170
+
164
171
To create the package follow the standard python setup.py to compile.
165
172
To test, just execute the python tests within the test folder
166
173
167
- Why using an appender rather than logstash or beats
168
- ---------------------------------------------------
174
+ ## Why using an appender rather than logstash or beats
175
+
169
176
In some cases is quite useful to provide all the information available within the LogRecords as it contains
170
177
things such as exception information, the method, file, log line where the log was generated.
171
178
@@ -177,29 +184,6 @@ using `SysLogHandler <https://docs.python.org/3/library/logging.handlers.html#sy
177
184
` logstash syslog plugin <https://www.elastic.co/guide/en/logstash/current/plugins-inputs-syslog.html> ` _ .
178
185
179
186
180
- Contributing back
181
- -----------------
182
- Feel free to use this as is or even better, feel free to fork and send your pull requests over.
183
-
184
-
185
- .. |downloads | image :: https://img.shields.io/pypi/dd/CMRESHandler.svg
186
- :target: https://pypi.python.org/pypi/CMRESHandler
187
- :alt: Daily PyPI downloads
188
- .. |versions | image :: https://img.shields.io/pypi/pyversions/CMRESHandler.svg
189
- :target: https://pypi.python.org/pypi/CMRESHandler
190
- :alt: Python versions supported
191
- .. |status | image :: https://img.shields.io/pypi/status/CMRESHandler.svg
192
- :target: https://pypi.python.org/pypi/CMRESHandler
193
- :alt: Package stability
194
- .. |license | image :: https://img.shields.io/pypi/l/CMRESHandler.svg
195
- :target: https://pypi.python.org/pypi/CMRESHandler
196
- :alt: License
197
- .. |ci_status | image :: https://travis-ci.org/cmanaha/python-elasticsearch-logger.svg?branch=master
198
- :target: https://travis-ci.org/cmanaha/python-elasticsearch-logger
199
- :alt: Continuous Integration Status
200
- .. |codecov | image :: https://codecov.io/github/cmanaha/python-elasticsearch-logger/coverage.svg?branch=master
201
- :target: http://codecov.io/github/cmanaha/python-elasticsearch-logger?branch=master
202
- :alt: Coverage!
203
- .. |gitter | image :: https://badges.gitter.im/Join%20Chat.svg
204
- :target: https://gitter.im/cmanaha/python-elasticsearch-logger?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge
205
- :alt: gitter
187
+ ## Contributing back
188
+
189
+ Feel free to use this as is or even better, feel free to fork and send your pull requests over.
0 commit comments