Skip to content

Support for multiple versions of the files with fallback to default version. #44

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 49 additions & 0 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,55 @@ authenticate use of database *my_app* with username/password combo *foo/bar*.
The gridfs root_collection is specified as *pics*. Nginx will then serve the
file in gridfs with _id *123...* for any request to */gridfs/123...*

Multiple versions of the files
===============================

**nginx-gridfs** allows you to serve multiple versions of the same file based on a query parameter.
This is useful, for instance, when you want to serve images of different sizes and/or quality.

To use this, you just have to provide a *type=<value>* query parameter when requesting the file.
**nginx-gridfs** will pick it up automatically, append the *value* to the filename with an underscore
(_) in between and serve that particular version of the file.

**NOTE** :

* This can be used only if the field type that is being queried on is String. This will not work with ObjectId and Integer fields.
* The name of the query parameter **must** be *type*.
* The additional versions of the file **must** follow the convention of naming the file as *filename_<version>*

Examples
---------

Assuming you have the following configuration to serve images from GridFS::

location /gridfs/ {
gridfs my_app field=filename type=string;
mongo 127.0.0.1:27017;
}

and you two extra versions of the images named *small* and *mid*.

* /gridfs/foo will serve the default version of the file with the filename *foo*
* /gridfs/foo?type=small will serve the "small" version of the file with the filename *foo_small*
* /gridfs/foo?type=mid will serve the "mid" version of the file with the filename *foo_mid*
* /gridfs/foo?type=not-existing will serve the default of the file with the filename *foo* - Read the next secion for details.


Fallback to default version
----------------------------

If a "type" is specified and that particular version of the file is not found, then *nginx-gridfs* will
fallback to the default version of the file and serve that instead. This is particularly useful if you
are generating the additional versions of the files asynchronously and all the versions may not always
be available. This will also allow you to discard any unknown versions requested and serve the default
version for all those requests.

In the above examples if */gridfs/foo?type=small* is requested and the file *foo_small* is not present
in GridFS, **nginx-gridfs** will instead serve the default version of the file *foo*.

Later when the *foo_small* version of the file is added, any subsequent request for */gridfs/foo?type=small*
will be served the small version.

Known Issues / TODO / Things You Should Hack On
===============================================

Expand Down
107 changes: 84 additions & 23 deletions ngx_http_gridfs_module.c
Original file line number Diff line number Diff line change
Expand Up @@ -673,6 +673,10 @@ static ngx_int_t ngx_http_gridfs_handler(ngx_http_request_t* request) {
ngx_str_t location_name;
ngx_str_t full_uri;
char* value;
ngx_str_t type_param;
ngx_flag_t valid_type = 0;
ngx_flag_t find_untyped = 1;
char* filename_with_type;
ngx_http_mongo_connection_t *mongo_conn;
gridfs gfs;
gridfile gfile;
Expand All @@ -685,6 +689,7 @@ static ngx_int_t ngx_http_gridfs_handler(ngx_http_request_t* request) {
volatile ngx_uint_t i;
ngx_int_t rc = NGX_OK;
bson query;
bson query_with_type;
bson_oid_t oid;
mongo_cursor ** cursors;
gridfs_offset chunk_len;
Expand Down Expand Up @@ -729,7 +734,8 @@ static ngx_int_t ngx_http_gridfs_handler(ngx_http_request_t* request) {
return NGX_HTTP_INTERNAL_SERVER_ERROR;
}

value = (char*)malloc(sizeof(char) * (full_uri.len - location_name.len + 1));
int value_len = full_uri.len - location_name.len + 1;
value = (char*)malloc(sizeof(char) * value_len);
if (value == NULL) {
ngx_log_error(NGX_LOG_ERR, request->connection->log, 0,
"Failed to allocate memory for value buffer.");
Expand All @@ -745,6 +751,32 @@ static ngx_int_t ngx_http_gridfs_handler(ngx_http_request_t* request) {
return NGX_HTTP_BAD_REQUEST;
}

/*
* Check if a "type" parameter has been specified, asking for a specific version of
* the file. If yes we append the value of the parameter to the filename with an
* underscore in between to be used for querying.
*
* We keep the existing filename as it is so that we can fallback to that in case the
* specified version of the file is not present
*
* NOTE : This "type" parameter will be useful only if the type of the query field is
* String. ObjectId and Integer fields cannot leverage this.
*/
if(ngx_http_arg(request, (u_char *) "type", 4, &type_param) == NGX_OK) {
valid_type = 1;
/* We need one extra byte for the _ (underscore) between filename and type */
filename_with_type = (char*)malloc(sizeof(char) * (value_len + type_param.len + 1));
if (filename_with_type == NULL) {
ngx_log_error(NGX_LOG_ERR, request->connection->log, 0,
"Failed to allocate memory for filename_with_type buffer.");
return NGX_HTTP_INTERNAL_SERVER_ERROR;
}
memcpy(filename_with_type, value, value_len - 1);
filename_with_type[value_len - 1] = '_';
memcpy(filename_with_type + value_len, type_param.data, type_param.len);
filename_with_type[value_len + type_param.len] = '\0';
}

// ---------- RETRIEVE GRIDFILE ---------- //

do {
Expand All @@ -766,29 +798,58 @@ static ngx_int_t ngx_http_gridfs_handler(ngx_http_request_t* request) {
}
} while (e);

bson_init(&query);
switch (gridfs_conf->type) {
case BSON_OID:
bson_oid_from_string(&oid, value);
bson_append_oid(&query, (char*)gridfs_conf->field.data, &oid);
break;
case BSON_INT:
bson_append_int(&query, (char*)gridfs_conf->field.data, ngx_atoi((u_char*)value, strlen(value)));
break;
case BSON_STRING:
bson_append_string(&query, (char*)gridfs_conf->field.data, value);
break;
}
bson_finish(&query);

status = gridfs_find_query(&gfs, &query, &gfile);

bson_destroy(&query);
free(value);
/*
* If we are provided with a "type" parameter then search for that specific version of the file
* first. If the specific version was found set the find_untyped to 0 (false) to prevent the
* fallback code from searching the default version of the file.
*
* If the search for the file resulted in an error the fallback code will be kick in automatically
* and search for the default version.
*/
if (valid_type && filename_with_type != NULL) {
bson_init(&query_with_type);
bson_append_string(&query_with_type, (char*)gridfs_conf->field.data, filename_with_type);
bson_finish(&query_with_type);

if(status == MONGO_ERROR) {
gridfs_destroy(&gfs);
return NGX_HTTP_NOT_FOUND;
status = gridfs_find_query(&gfs, &query_with_type, &gfile);

bson_destroy(&query_with_type);
free(filename_with_type);

if(status != MONGO_ERROR) {
find_untyped = 0;
}
}

/*
* Search the for the default version of the file only if a specific version was not specified
* or the specified version of the file was not found.
*/
if (find_untyped) {
bson_init(&query);
switch (gridfs_conf->type) {
case BSON_OID:
bson_oid_from_string(&oid, value);
bson_append_oid(&query, (char*)gridfs_conf->field.data, &oid);
break;
case BSON_INT:
bson_append_int(&query, (char*)gridfs_conf->field.data, ngx_atoi((u_char*)value, strlen(value)));
break;
case BSON_STRING:
bson_append_string(&query, (char*)gridfs_conf->field.data, value);
break;
}
bson_finish(&query);

status = gridfs_find_query(&gfs, &query, &gfile);

bson_destroy(&query);
free(value);

if(status == MONGO_ERROR) {
gridfs_destroy(&gfs);
return NGX_HTTP_NOT_FOUND;
}
}

/* Get information about the file */
Expand Down