Skip to content

Commit 72d56bd

Browse files
dimon222kevin-bates
authored andcommitted
More details for readme (#53)
* More details for readme * README to markdown
1 parent 418c710 commit 72d56bd

File tree

3 files changed

+121
-126
lines changed

3 files changed

+121
-126
lines changed

README.md

Lines changed: 120 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,120 @@
1+
# hadoop-yarn-api-python-client
2+
3+
Python client for Apache Hadoop® YARN API
4+
5+
[![Latest
6+
Version](https://img.shields.io/pypi/v/yarn-api-client.svg)](https://pypi.python.org/pypi/yarn-api-client/) [![Travis CI build
7+
status](https://travis-ci.org/toidi/hadoop-yarn-api-python-client.svg?branch=master)](https://travis-ci.org/toidi/hadoop-yarn-api-python-client) [![Latest documentation
8+
status](http://readthedocs.org/projects/python-client-for-hadoop-yarn-api/badge/?version=latest)](https://python-client-for-hadoop-yarn-api.readthedocs.org/en/latest/?badge=latest) [![Test
9+
coverage](https://coveralls.io/repos/toidi/hadoop-yarn-api-python-client/badge.png)](https://coveralls.io/r/toidi/hadoop-yarn-api-python-client)
10+
11+
Package documentation:
12+
[python-client-for-hadoop-yarn-api.readthedocs.org](http://python-client-for-hadoop-yarn-api.readthedocs.org/en/latest/)
13+
14+
REST API documentation: [hadoop.apache.org](http://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/WebServicesIntro.html)
15+
16+
---
17+
## Compatibility
18+
Library is compatible with Apache Hadoop __**3.2.1**__.
19+
20+
If u have version other than mentioned (or vendored variant like Hortonworks), certain APIs might be not working or have differences in
21+
implementation. If u plan to use certain API long-term, you might want to make sure its not in Alpha stage in documentation.
22+
23+
## Installation
24+
25+
From PyPI
26+
```
27+
pip install yarn-api-client
28+
```
29+
30+
From Anaconda (conda forge)
31+
```
32+
conda install -c conda-forge yarn-api-client
33+
```
34+
35+
From source code
36+
```
37+
git clone https://github.com/toidi/hadoop-yarn-api-python-client.git
38+
pushd hadoop-yarn-api-python-client
39+
python setup.py install
40+
popd
41+
```
42+
43+
## Enabling support for Kerberos/SPNEGO Security
44+
1. First option - using `requests_kerberos` package
45+
46+
To avoid deployment issues on a non Kerberized environment, the `requests_kerberos`
47+
dependency is optional and needs to be explicit installed in order to enable access
48+
to YARN console protected by Kerberos/SPNEGO.
49+
50+
`pip install requests_kerberos`
51+
52+
From python code
53+
```
54+
from yarn_api_client.history_server import HistoryServer
55+
from requests_kerberos import HTTPKerberosAuth
56+
history_server = HistoryServer('https://127.0.0.2:5678', auth=HTTPKerberosAuth())
57+
```
58+
59+
PS: You __**need**__ to get valid kerberos ticket in systemwide kerberos cache before running your code, otherwise calls to kerberized environment won't go through (run kinit before proceeding to run code)
60+
61+
2. Second option - using `gssapi` package
62+
63+
If you want to avoid using terminal calls, you have to perform SPNEGO handshake to retrieve ticket yourself. Full API documentation: https://pythongssapi.github.iQ/python-gssapi/latest/_
64+
65+
# Usage
66+
67+
### CLI interface
68+
69+
1. First way
70+
```
71+
bin/yarn_client --help
72+
```
73+
74+
2. Alternative way
75+
```
76+
python -m yarn_api_client --help
77+
```
78+
79+
### Programmatic interface
80+
81+
```
82+
from yarn_api_client import ApplicationMaster, HistoryServer, NodeManager, ResourceManager
83+
am = ApplicationMaster('https://127.0.0.2:5678')
84+
app_information = am.application_information('application_id')
85+
```
86+
87+
### Changelog
88+
89+
0.3.6 Release
90+
- Extend ResourceManager to allow appli
91+
determine resource availability prior to submission.
92+
93+
0.3.5 Release
94+
- Hotfix release to fix internal signature mismatch
95+
96+
0.3.4 Release
97+
- More flexible support for discovering Hadoop configuration
98+
including multiple Resource Managers when HA is configured
99+
- Properly support YARN post response codes
100+
101+
0.3.3 Release
102+
- Properly set Content-Type in PUT requests
103+
- Check for HADOOP_CONF_DIR env variable
104+
105+
0.3.2 Release
106+
- Make Kerberos/SPNEGO dependency optional
107+
108+
0.3.1 Release
109+
- Fix cluster_application_kill API
110+
111+
0.3.0 Release
112+
- Add support for YARN endpoints protected by Kerberos/SPNEGO
113+
- Moved to `requests` package for REST API invocation
114+
- Remove `http_con` property, as connections are now managed by `requests` package
115+
116+
0.2.5 Release
117+
- Fixed History REST API
118+
119+
0.2.4 Release
120+
- Added compatibility with HA enabled Resource Manager

README.rst

Lines changed: 0 additions & 125 deletions
This file was deleted.

setup.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ def find_version(*file_paths):
2323
name = 'yarn-api-client',
2424
version = find_version('yarn_api_client', '__init__.py'),
2525
description='Python client for Hadoop® YARN API',
26-
long_description=read('README.rst'),
26+
long_description=read('README.md'),
2727
packages = find_packages(exclude=['tests','itests']),
2828

2929
install_requires = [

0 commit comments

Comments
 (0)