Merge pull request encode#3313 from tomchristie/limit-selects

Limit rendering of relational selects to max 1000 items by default.

Author: tomchristie

docs/api-guide/fields.md

@@ -100,7 +100,7 @@ Two options are currently used in HTML form generation, `'input_type'` and `'bas
         style = {'base_template': 'radio.html'}
     }
 
-**Note**: The `style` argument replaces the old-style version 2.x `widget` keyword argument. Because REST framework 3 now uses templated HTML form generation, the `widget` option that was used to support Django built-in widgets can no longer be supported. Version 3.1 is planned to include public API support for customizing HTML form generation.
+**Note**: The `style` argument replaces the old-style version 2.x `widget` keyword argument. Because REST framework 3 now uses templated HTML form generation, the `widget` option that was used to support Django built-in widgets can no longer be supported. Version 3.3 is planned to include public API support for customizing HTML form generation.
 
 ---
 
@@ -364,6 +364,8 @@ Used by `ModelSerializer` to automatically generate fields if the corresponding
 
 - `choices` - A list of valid values, or a list of `(key, display_name)` tuples.
 - `allow_blank` - If set to `True` then the empty string should be considered a valid value. If set to `False` then the empty string is considered invalid and will raise a validation error. Defaults to `False`.
+- `html_cutoff` - If set this will be the maximum number of choices that will be displayed by a HTML select drop down. Can be used to ensure that automatically generated ChoiceFields with very large possible selections do not prevent a template from rendering. Defaults to `None`.
+- `html_cutoff_text` - If set this will display a textual indicator if the maximum number of items have been cutoff in an HTML select drop down. Defaults to `"More than {count} items…"`
 
 Both the `allow_blank` and `allow_null` are valid options on `ChoiceField`, although it is highly recommended that you only use one and not both. `allow_blank` should be preferred for textual choices, and `allow_null` should be preferred for numeric or other non-textual choices.
 
@@ -375,6 +377,8 @@ A field that can accept a set of zero, one or many values, chosen from a limited
 
 - `choices` - A list of valid values, or a list of `(key, display_name)` tuples.
 - `allow_blank` - If set to `True` then the empty string should be considered a valid value. If set to `False` then the empty string is considered invalid and will raise a validation error. Defaults to `False`.
+- `html_cutoff` - If set this will be the maximum number of choices that will be displayed by a HTML select drop down. Can be used to ensure that automatically generated ChoiceFields with very large possible selections do not prevent a template from rendering. Defaults to `None`.
+- `html_cutoff_text` - If set this will display a textual indicator if the maximum number of items have been cutoff in an HTML select drop down. Defaults to `"More than {count} items…"`
 
 As with `ChoiceField`, both the `allow_blank` and `allow_null` options are valid, although it is highly recommended that you only use one and not both. `allow_blank` should be preferred for textual choices, and `allow_null` should be preferred for numeric or other non-textual choices.
 

docs/api-guide/relations.md

@@ -16,7 +16,7 @@ Relational fields are used to represent model relationships.  They can be applie
 
 ---
 
-#### Inspecting automatically generated relationships.
+#### Inspecting relationships.
 
 When using the `ModelSerializer` class, serializer fields and relationships will be automatically generated for you. Inspecting these automatically generated fields can be a useful tool for determining how to customize the relationship style.
 
@@ -442,6 +442,25 @@ To provide customized representations for such inputs, override `display_value()
         def display_value(self, instance):
             return 'Track: %s' % (instance.title)
 
+## Select field cutoffs
+
+When rendered in the browsable API relational fields will default to only displaying a maximum of 1000 selectable items. If more items are present then a disabled option with "More than 1000 items…" will be displayed.
+
+This behavior is intended to prevent a template from being unable to render in an acceptable timespan due to a very large number of relationships being displayed.
+
+There are two keyword arguments you can use to control this behavior:
+
+- `html_cutoff` - If set this will be the maximum number of choices that will be displayed by a HTML select drop down. Set to `None` to disable any limiting. Defaults to `1000`.
+- `html_cutoff_text` - If set this will display a textual indicator if the maximum number of items have been cutoff in an HTML select drop down. Defaults to `"More than {count} items…"`
+
+In cases where the cutoff is being enforced you may want to instead use a plain input field in the HTML form. You can do so using the `style` keyword argument. For example:
+
+    assigned_to = serializers.SlugRelatedField(
+       queryset=User.objects.all(),
+       slug field='username',
+       style={'base_template': 'input.html'}
+    )
+
 ## Reverse relations
 
 Note that reverse relationships are not automatically included by the `ModelSerializer` and `HyperlinkedModelSerializer` classes.  To include a reverse relationship, you must explicitly add it to the fields list.  For example:

rest_framework/fields.py

@@ -156,7 +156,7 @@ def flatten_choices_dict(choices):
     return ret
 
 
-def iter_options(grouped_choices):
+def iter_options(grouped_choices, cutoff=None, cutoff_text=None):
     """
     Helper function for options and option groups in templates.
     """
@@ -175,18 +175,32 @@ class Option(object):
         start_option_group = False
         end_option_group = False
 
-        def __init__(self, value, display_text):
+        def __init__(self, value, display_text, disabled=False):
             self.value = value
             self.display_text = display_text
+            self.disabled = disabled
+
+    count = 0
 
     for key, value in grouped_choices.items():
+        if cutoff and count >= cutoff:
+            break
+
         if isinstance(value, dict):
             yield StartOptionGroup(label=key)
             for sub_key, sub_value in value.items():
+                if cutoff and count >= cutoff:
+                    break
                 yield Option(value=sub_key, display_text=sub_value)
+                count += 1
             yield EndOptionGroup()
         else:
             yield Option(value=key, display_text=value)
+            count += 1
+
+    if cutoff and count >= cutoff and cutoff_text:
+        cutoff_text = cutoff_text.format(count=cutoff)
+        yield Option(value='n/a', display_text=cutoff_text, disabled=True)
 
 
 class CreateOnlyDefault(object):
@@ -1188,10 +1202,14 @@ class ChoiceField(Field):
     default_error_messages = {
         'invalid_choice': _('"{input}" is not a valid choice.')
     }
+    html_cutoff = None
+    html_cutoff_text = _('More than {count} items...')
 
     def __init__(self, choices, **kwargs):
         self.grouped_choices = to_choices_dict(choices)
         self.choices = flatten_choices_dict(self.grouped_choices)
+        self.html_cutoff = kwargs.pop('html_cutoff', self.html_cutoff)
+        self.html_cutoff_text = kwargs.pop('html_cutoff_text', self.html_cutoff_text)
 
         # Map the string representation of choices to the underlying value.
         # Allows us to deal with eg. integer choices while supporting either
@@ -1222,7 +1240,11 @@ def iter_options(self):
         """
         Helper method for use with templates rendering select widgets.
         """
-        return iter_options(self.grouped_choices)
+        return iter_options(
+            self.grouped_choices,
+            cutoff=self.html_cutoff,
+            cutoff_text=self.html_cutoff_text
+        )
 
 
 class MultipleChoiceField(ChoiceField):

rest_framework/relations.py

@@ -54,9 +54,13 @@ def __init__(self, pk):
 
 class RelatedField(Field):
     queryset = None
+    html_cutoff = 1000
+    html_cutoff_text = _('More than {count} items...')
 
     def __init__(self, **kwargs):
         self.queryset = kwargs.pop('queryset', self.queryset)
+        self.html_cutoff = kwargs.pop('html_cutoff', self.html_cutoff)
+        self.html_cutoff_text = kwargs.pop('html_cutoff_text', self.html_cutoff_text)
         assert self.queryset is not None or kwargs.get('read_only', None), (
             'Relational field must provide a `queryset` argument, '
             'or set read_only=`True`.'
@@ -158,7 +162,11 @@ def grouped_choices(self):
         return self.choices
 
     def iter_options(self):
-        return iter_options(self.grouped_choices)
+        return iter_options(
+            self.grouped_choices,
+            cutoff=self.html_cutoff,
+            cutoff_text=self.html_cutoff_text
+        )
 
     def display_value(self, instance):
         return six.text_type(instance)
@@ -415,10 +423,15 @@ class ManyRelatedField(Field):
         'not_a_list': _('Expected a list of items but got type "{input_type}".'),
         'empty': _('This list may not be empty.')
     }
+    html_cutoff = 1000
+    html_cutoff_text = _('More than {count} items...')
 
     def __init__(self, child_relation=None, *args, **kwargs):
         self.child_relation = child_relation
         self.allow_empty = kwargs.pop('allow_empty', True)
+        self.html_cutoff = kwargs.pop('html_cutoff', self.html_cutoff)
+        self.html_cutoff_text = kwargs.pop('html_cutoff_text', self.html_cutoff_text)
+
         assert child_relation is not None, '`child_relation` is a required argument.'
         super(ManyRelatedField, self).__init__(*args, **kwargs)
         self.child_relation.bind(field_name='', parent=self)
@@ -469,4 +482,8 @@ def grouped_choices(self):
         return self.choices
 
     def iter_options(self):
-        return iter_options(self.grouped_choices)
+        return iter_options(
+            self.grouped_choices,
+            cutoff=self.html_cutoff,
+            cutoff_text=self.html_cutoff_text
+        )

rest_framework/templates/rest_framework/horizontal/select.html

@@ -16,7 +16,7 @@
           {% elif select.end_option_group %}
             </optgroup>
           {% else %}
-            <option value="{{ select.value }}" {% if select.value == field.value %}selected{% endif %}>{{ select.display_text }}</option>
+            <option value="{{ select.value }}" {% if select.value == field.value %}selected{% endif %} {% if select.disabled %}disabled{% endif %}>{{ select.display_text }}</option>
           {% endif %}
       {% endfor %}
     </select>

rest_framework/templates/rest_framework/horizontal/select_multiple.html

@@ -16,7 +16,7 @@
         {% elif select.end_option_group %}
           </optgroup>
         {% else %}
-          <option value="{{ select.value }}" {% if select.value in field.value %}selected{% endif %}>{{ select.display_text }}</option>
+          <option value="{{ select.value }}" {% if select.value in field.value %}selected{% endif %} {% if select.disabled %}disabled{% endif %}>{{ select.display_text }}</option>
         {% endif %}
       {% empty %}
           <option>{{ no_items }}</option>

rest_framework/templates/rest_framework/inline/select.html

@@ -15,7 +15,7 @@
         {% elif select.end_option_group %}
           </optgroup>
         {% else %}
-          <option value="{{ select.value }}" {% if select.value == field.value %}selected{% endif %}>{{ select.display_text }}</option>
+          <option value="{{ select.value }}" {% if select.value == field.value %}selected{% endif %} {% if select.disabled %}disabled{% endif %}>{{ select.display_text }}</option>
         {% endif %}
     {% endfor %}
   </select>

rest_framework/templates/rest_framework/inline/select_multiple.html

@@ -15,7 +15,7 @@
           {% elif select.end_option_group %}
             </optgroup>
           {% else %}
-            <option value="{{ select.value }}" {% if select.value in field.value %}selected{% endif %}>{{ select.display_text }}</option>
+            <option value="{{ select.value }}" {% if select.value in field.value %}selected{% endif %} {% if select.disabled %}disabled{% endif %}>{{ select.display_text }}</option>
           {% endif %}
       {% empty %}
       <option>{{ no_items }}</option>

rest_framework/templates/rest_framework/vertical/select.html

@@ -15,7 +15,7 @@
         {% elif select.end_option_group %}
           </optgroup>
         {% else %}
-          <option value="{{ select.value }}" {% if select.value == field.value %}selected{% endif %}>{{ select.display_text }}</option>
+          <option value="{{ select.value }}" {% if select.value == field.value %}selected{% endif %} {% if select.disabled %}disabled{% endif %}>{{ select.display_text }}</option>
         {% endif %}
     {% endfor %}
   </select>

rest_framework/templates/rest_framework/vertical/select_multiple.html

@@ -15,7 +15,7 @@
         {% elif select.end_option_group %}
           </optgroup>
         {% else %}
-          <option value="{{ select.value }}" {% if select.value in field.value %}selected{% endif %}>{{ select.display_text }}</option>
+          <option value="{{ select.value }}" {% if select.value in field.value %}selected{% endif %} {% if select.disabled %}disabled{% endif %}>{{ select.display_text }}</option>
         {% endif %}
     {% empty %}
         <option>{{ no_items }}</option>