1
-
2
- ===============
3
- CMRESHandler.py
4
- ===============
5
-
6
- | |license| |versions| |status| |downloads|
7
- | |ci_status| |codecov| |gitter|
8
-
9
-
1
+ @@ -0,0 +1,190 @@
2
+ @@ -0,0 +1,189 @@
10
3
** Note: Maintainers needed : Those that committed in the past code to this repo or are presenting new PRs and have experience and interest on helping to maintain repos & python libraries (code quality, testing, integration, etc). If you are intereted on getting our PR's through and helping others to contribute to the library, please get in touch.**
11
4
5
+ ----
12
6
13
- Python Elasticsearch Log handler
14
- ********************************
7
+ [ ![ downloads] ( https://img.shields.io/pypi/dd/CMRESHandler.svg )] ( https://pypi.python.org/pypi/CMRESHandler )
8
+ [ ![ versions] ( https://img.shields.io/pypi/pyversions/CMRESHandler.svg )] ( https://pypi.python.org/pypi/CMRESHandler )
9
+ [ ![ status] ( https://img.shields.io/pypi/status/CMRESHandler.svg )] ( https://pypi.python.org/pypi/CMRESHandler )
10
+ [ ![ license] ( https://img.shields.io/pypi/l/CMRESHandler.svg )] ( https://pypi.python.org/pypi/CMRESHandler )
11
+ [ ![ ci_status] ( https://travis-ci.org/cmanaha/python-elasticsearch-logger.svg?branch=master )] ( https://travis-ci.org/cmanaha/python-elasticsearch-logger )
12
+ [ ![ codecov] ( https://codecov.io/github/cmanaha/python-elasticsearch-logger/coverage.svg?branch=master )] ( http://codecov.io/github/cmanaha/python-elasticsearch-logger?branch=master )
13
+ [ ![ gitter] ( https://badges.gitter.im/Join%20Chat.svg )] ( https://gitter.im/cmanaha/python-elasticsearch-logger?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge )
14
+
15
+ # Python Elasticsearch Log handler
15
16
16
17
This library provides an Elasticsearch logging appender compatible with the
17
18
python standard ` logging <https://docs.python.org/2/library/logging.html> ` _ library.
@@ -20,58 +21,62 @@ The code source is in github at `https://github.com/cmanaha/python-elasticsearch
20
21
< https://github.com/cmanaha/python-elasticsearch-logger > `_
21
22
22
23
23
- Installation
24
- ============
24
+ # Installation
25
+
25
26
Install using pip::
26
27
27
28
pip install CMRESHandler
28
29
29
- Requirements Python 2
30
- =====================
30
+ # Requirements Python 2
31
+
31
32
This library requires the following dependencies
32
33
- elasticsearch
33
34
- requests
34
35
- enum
35
36
36
37
37
- Requirements Python 3
38
- =====================
38
+ # Requirements Python 3
39
+
39
40
This library requires the following dependencies
40
41
- elasticsearch
41
42
- requests
42
43
43
- Additional requirements for Kerberos support
44
- ============================================
45
- Additionally, the package support optionally kerberos authentication by adding the following dependecy
44
+ # Additional requirements for Kerberos support
45
+
46
+ Additionally, the package support optionally kerberos authentication by adding the following dependency
46
47
- requests-kerberos
47
48
48
- Additional requirements for AWS IAM user authentication (request signing)
49
- =========================================================================
50
- Additionally, the package support optionally AWS IAM user authentication by adding the following dependecy
49
+ # Additional requirements for AWS IAM user authentication (request signing)
50
+
51
+ Additionally, the package support optionally AWS IAM user authentication by adding the following dependency
51
52
- requests-aws4auth
52
53
53
- Using the handler in your program
54
- ==================================
55
- To initialise and create the handler, just add the handler to your logger as follow ::
54
+ # Using the handler in your program
55
+
56
+ To initialize and create the handler, just add the handler to your logger as follow ::
56
57
57
- from cmreslogging.handlers import CMRESHandler
58
- handler = CMRESHandler(hosts=[{'host': 'localhost', 'port': 9200}],
59
- auth_type=CMRESHandler.AuthType.NO_AUTH,
60
- es_index_name="my_python_index")
61
- log = logging.getLogger("PythonTest")
62
- log.setLevel(logging.INFO)
63
- log.addHandler(handler)
58
+ ``` python
59
+ from cmreslogging.handlers import CMRESHandler
60
+ handler = CMRESHandler(hosts = [{' host' : ' localhost' , ' port' : 9200 }],
61
+ auth_type = CMRESHandler.AuthType.NO_AUTH ,
62
+ es_index_name = " my_python_index" )
63
+ log = logging.getLogger(" PythonTest" )
64
+ log.setLevel(logging.INFO )
65
+ log.addHandler(handler)
66
+ ```
64
67
65
68
You can add fields upon initialisation, providing more data of the execution context ::
66
69
67
- from cmreslogging.handlers import CMRESHandler
68
- handler = CMRESHandler(hosts=[{'host': 'localhost', 'port': 9200}],
69
- auth_type=CMRESHandler.AuthType.NO_AUTH,
70
- es_index_name="my_python_index",
71
- es_additional_fields={'App': 'MyAppName', 'Environment': 'Dev'})
72
- log = logging.getLogger("PythonTest")
73
- log.setLevel(logging.INFO)
74
- log.addHandler(handler)
70
+ ``` python
71
+ from cmreslogging.handlers import CMRESHandler
72
+ handler = CMRESHandler(hosts = [{' host' : ' localhost' , ' port' : 9200 }],
73
+ auth_type = CMRESHandler.AuthType.NO_AUTH ,
74
+ es_index_name = " my_python_index" ,
75
+ es_additional_fields = {' App' : ' MyAppName' , ' Environment' : ' Dev' })
76
+ log = logging.getLogger(" PythonTest" )
77
+ log.setLevel(logging.INFO )
78
+ log.addHandler(handler)
79
+ ```
75
80
76
81
This additional fields will be applied to all logging fields and recorded in elasticsearch
77
82
@@ -82,18 +87,20 @@ To log, use the regular commands from the logging library ::
82
87
Your code can also dump additional extra fields on a per log basis that can be used to instrument
83
88
operations. For example, when reading information from a database you could do something like::
84
89
85
- start_time = time.time()
86
- database_operation()
87
- db_delta = time.time() - start_time
88
- log.debug("DB operation took %.3f seconds" % db_delta, extra={'db_execution_time': db_delta})
90
+ ``` python
91
+ start_time = time.time()
92
+ database_operation()
93
+ db_delta = time.time() - start_time
94
+ log.debug(" DB operation took %.3f seconds" % db_delta, extra = {' db_execution_time' : db_delta})
95
+ ```
89
96
90
97
The code above executes the DB operation, measures the time it took and logs an entry that contains
91
98
in the message the time the operation took as string and for convenience, it creates another field
92
99
called db_execution_time with a float that can be used to plot the time this operations are taking using
93
100
Kibana on top of elasticsearch
94
101
95
- Initialisation parameters
96
- =========================
102
+ # Initialization parameters
103
+
97
104
The constructors takes the following parameters:
98
105
- hosts: The list of hosts that elasticsearch clients will connect, multiple hosts are allowed, for example ::
99
106
@@ -118,54 +125,56 @@ The constructors takes the following parameters:
118
125
- es_doc_type: A string with the name of the document type that will be used `` python_log `` used by default
119
126
- es_additional_fields: A dictionary with all the additional fields that you would like to add to the logs
120
127
121
- Django Integration
122
- ==================
128
+ # Django Integration
129
+
123
130
It is also very easy to integrate the handler to ` Django <https://www.djangoproject.com/> ` _ And what is even
124
131
better, at DEBUG level django logs information such as how long it takes for DB connections to return so
125
132
they can be plotted on Kibana, or the SQL statements that Django executed. ::
126
133
127
- from cmreslogging.handlers import CMRESHandler
128
- LOGGING = {
129
- 'version': 1,
130
- 'disable_existing_loggers': False,
131
- 'handlers': {
132
- 'file': {
133
- 'level': 'DEBUG',
134
- 'class': 'logging.handlers.RotatingFileHandler',
135
- 'filename': './debug.log',
136
- 'maxBytes': 102400,
137
- 'backupCount': 5,
138
- },
139
- 'elasticsearch': {
140
- 'level': 'DEBUG',
141
- 'class': 'cmreslogging.handlers.CMRESHandler',
142
- 'hosts': [{'host': 'localhost', 'port': 9200}],
143
- 'es_index_name': 'my_python_app',
144
- 'es_additional_fields': {'App': 'Test', 'Environment': 'Dev'},
145
- 'auth_type': CMRESHandler.AuthType.NO_AUTH,
146
- 'use_ssl': False,
147
- },
134
+ ``` python
135
+ from cmreslogging.handlers import CMRESHandler
136
+ LOGGING = {
137
+ ' version' : 1 ,
138
+ ' disable_existing_loggers' : False ,
139
+ ' handlers' : {
140
+ ' file' : {
141
+ ' level' : ' DEBUG' ,
142
+ ' class' : ' logging.handlers.RotatingFileHandler' ,
143
+ ' filename' : ' ./debug.log' ,
144
+ ' maxBytes' : 102400 ,
145
+ ' backupCount' : 5 ,
148
146
},
149
- 'loggers': {
150
- 'django': {
151
- 'handlers': ['file','elasticsearch'],
152
- 'level': 'DEBUG',
153
- 'propagate': True,
154
- },
147
+ ' elasticsearch' : {
148
+ ' level' : ' DEBUG' ,
149
+ ' class' : ' cmreslogging.handlers.CMRESHandler' ,
150
+ ' hosts' : [{' host' : ' localhost' , ' port' : 9200 }],
151
+ ' es_index_name' : ' my_python_app' ,
152
+ ' es_additional_fields' : {' App' : ' Test' , ' Environment' : ' Dev' },
153
+ ' auth_type' : CMRESHandler.AuthType.NO_AUTH ,
154
+ ' use_ssl' : False ,
155
155
},
156
- }
156
+ },
157
+ ' loggers' : {
158
+ ' django' : {
159
+ ' handlers' : [' file' ,' elasticsearch' ],
160
+ ' level' : ' DEBUG' ,
161
+ ' propagate' : True ,
162
+ },
163
+ },
164
+ }
165
+ ```
157
166
158
167
There is more information about how Django logging works in the
159
168
` Django documentation <https://docs.djangoproject.com/en/1.9/topics/logging//> ` _
160
169
161
170
162
- Building the sources & Testing
163
- ------------------------------
171
+ ## Building the sources & Testing
172
+
164
173
To create the package follow the standard python setup.py to compile.
165
174
To test, just execute the python tests within the test folder
166
175
167
- Why using an appender rather than logstash or beats
168
- ---------------------------------------------------
176
+ ## Why using an appender rather than logstash or beats
177
+
169
178
In some cases is quite useful to provide all the information available within the LogRecords as it contains
170
179
things such as exception information, the method, file, log line where the log was generated.
171
180
@@ -177,29 +186,6 @@ using `SysLogHandler <https://docs.python.org/3/library/logging.handlers.html#sy
177
186
` logstash syslog plugin <https://www.elastic.co/guide/en/logstash/current/plugins-inputs-syslog.html> ` _ .
178
187
179
188
180
- Contributing back
181
- -----------------
182
- Feel free to use this as is or even better, feel free to fork and send your pull requests over.
183
-
184
-
185
- .. |downloads | image :: https://img.shields.io/pypi/dd/CMRESHandler.svg
186
- :target: https://pypi.python.org/pypi/CMRESHandler
187
- :alt: Daily PyPI downloads
188
- .. |versions | image :: https://img.shields.io/pypi/pyversions/CMRESHandler.svg
189
- :target: https://pypi.python.org/pypi/CMRESHandler
190
- :alt: Python versions supported
191
- .. |status | image :: https://img.shields.io/pypi/status/CMRESHandler.svg
192
- :target: https://pypi.python.org/pypi/CMRESHandler
193
- :alt: Package stability
194
- .. |license | image :: https://img.shields.io/pypi/l/CMRESHandler.svg
195
- :target: https://pypi.python.org/pypi/CMRESHandler
196
- :alt: License
197
- .. |ci_status | image :: https://travis-ci.org/cmanaha/python-elasticsearch-logger.svg?branch=master
198
- :target: https://travis-ci.org/cmanaha/python-elasticsearch-logger
199
- :alt: Continuous Integration Status
200
- .. |codecov | image :: https://codecov.io/github/cmanaha/python-elasticsearch-logger/coverage.svg?branch=master
201
- :target: http://codecov.io/github/cmanaha/python-elasticsearch-logger?branch=master
202
- :alt: Coverage!
203
- .. |gitter | image :: https://badges.gitter.im/Join%20Chat.svg
204
- :target: https://gitter.im/cmanaha/python-elasticsearch-logger?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge
205
- :alt: gitter
189
+ ## Contributing back
190
+
191
+ Feel free to use this as is or even better, feel free to fork and send your pull requests over.
0 commit comments