override_settings not working

[csdenboer]

In my unit tests, I'm facing troubles with requests being denied because of throttle rates. By setting DEFAULT_THROTTLE_CLASSES to (), requests are no longer throttled. However, I only want to do this for unit tests, so I tried to use override_settings to override the DEFAULT_THROTTLE_CLASSES setting.

    def test_login_invalid_email(self):
        with override_settings(REST_FRAMEWORK={**settings.REST_FRAMEWORK, 'DEFAULT_THROTTLE_CLASSES': ()}):
            data = {'email': 'invalid@email', 'password': self.data['password']}
            response = self.client.post(reverse('login'), data, secure=True)
            self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)

This does not work; I'm still getting 429's. How can I get this working?

Note: I'm using latest version of django-rest-framework.

[crccheck]

I'm running into this too, but for authentication. I'm guessing it's because of these lines:

django-rest-framework/rest_framework/views.py

Lines 96 to 104 in df77f7b

	# The following policies may be set at either globally, or per-view.
	renderer_classes = api_settings.DEFAULT_RENDERER_CLASSES
	parser_classes = api_settings.DEFAULT_PARSER_CLASSES
	authentication_classes = api_settings.DEFAULT_AUTHENTICATION_CLASSES
	throttle_classes = api_settings.DEFAULT_THROTTLE_CLASSES
	permission_classes = api_settings.DEFAULT_PERMISSION_CLASSES
	content_negotiation_class = api_settings.DEFAULT_CONTENT_NEGOTIATION_CLASS
	metadata_class = api_settings.DEFAULT_METADATA_CLASS
	versioning_class = api_settings.DEFAULT_VERSIONING_CLASS

Followup to #2466 ?

[rpkilby]

@crccheck that makes sense. If the class has already been defined, it has already accessed the relevant setting. Overriding the setting at a later date will have no effect on that view class.

We should probably set those values to None, then lookup the value from the api settings in the relevant methods. e.g., get_permissions or get_renderers.

For now, a workaround would be to use a settings module specifically for the test environment. e.g., disable throttling, setting password hashers to only MD5, etc...

[csdenboer]

This introduces the possibility of running tests with settings that differ from the production environment (apart from the ones that we want to override) which is really undesirable in my opinion.

[rpkilby]

This gets into a larger discussion about how to manage your settings, but our typical pattern is:

• have a single settings file for the application (usually just <project name>/settings.py
• all environment-specific values (eg, database URL) are loaded via the process's environment variables
  (django-environ, or manually import from os.environ).
• dev/test may use a separate settings module that minimally overrides the primary settings module. eg, in dev you probably want to use the console backend for sending emails to STDOUT. In test, you probably want to send emails via the locmem backend to test against mail.outbox.

With this pattern, our dev/test settings are usually ~5 lines of relatively inconsequential overrides. e.g,

# application/settings.py
DEBUG = os.environ.get("DEBUG") == "true"
ALL_THE_SETTINGS = "..."

# tests/settings.py
from application.settings import *
EMAIL_BACKEND = "..."

[crccheck]

In my case, I want some integration-style tests to run using the same DRF settings as production, but the rest to be more like unit-like tests where I'm trying to strip away boilerplate. I could do this with two test runs and different settings, but it'd be really nice if I could do this with override_settings.

It may be that adding special code to DRF just to support this case kills production performance. It'd make sense to not support it then. Maybe I can solve this with unittest.mock. This is my first DRF project so I'm just getting my feet wet.

[belugame]

@crccheck Have you gotten anywhere with this?
I want to write unittests for a extension library of DRF and need to test scenarios for SOME_SETTING being True and being False, but this issue stops me.

[crccheck]

alas no, I already wrote the boilerplate so I haven't been motivated to re-visit this. Maybe my next DRF project.

[tprestegard]

Oof, I lost half a day to this - adding a simple throttle to my project for anonymous users caused half of my unit tests to fail with 429 errors. I followed @crccheck's hint and just manually adjusted the throttle rate by mock.patching the get_rate method of the throttle. It was easy for me to do in one step, since all of my unit tests derive from a custom class, but might not be so easy for others.

[HansBambel]

For others that wondered how to mock this. This is how I did it:

import re
from unittest.mock import patch
from rest_framework.test import APIClient

@patch("rest_framework.throttling.AnonRateThrottle.get_rate", lambda x: "2/day")
def test_throttling():
    client = APIClient()
    for _ in range(2):
        resp = client.get("/v1/login")
        assert resp.status_code != 429

    resp = client.get("/v1/login")
    assert resp.status_code == 429, resp.json()
    assert re.match(r"Request was throttled. Expected available in \d+ seconds.", resp.json()["detail"])

[yamila-moreno]

Hi there; I'm running into this problem with api_settings.PAGE_SIZE; I'm using latest versions Django and DRF.

def test_pagination_with_users_and_pages(client, settings):
    # given
    settings.REST_FRAMEWORK['PAGE_SIZE'] = 2
    factories.create_users(num=5)

    # when
    response = client.get('/qapi/users/')

    # then
    assert 'next' in response
    assert response['next'] == 'http://testserver/qapi/users/?page=2'

Although I expected this to override the settings to test my custom paginator; after looking into the code, I got to understand why this is not working. settings fixture overrides Django settings, while DRF is using it's own instance of settings previously loaded.

Is there any way to make it work?

My second try was using a decorator in the test, so it would override the value before being called:

@override_settings(REST_FRAMEWORK={
    'PAGE_SIZE': 2,
    'DEFAULT_PAGINATION_CLASS': 'api.pagination.CustomHeaderPagination'
    })
def test_pagination_with_users_and_pages(client):
    # given
    factories.create_users(num=5)

    # when
    response = client.get('/qapi/users/')

    # then
    assert 'next' in response
    assert response['next'] == 'http://testserver/qapi/users/?page=2'

Again, looking into the code, this can't be expected. I made a few changes into class PageNumberPagination to solve this second scenario:

class PageNumberPagination(BasePagination):
-  page_size = api_settings.PAGE_SIZE
+  page_size = None

def get_page_size(self, request):                                       
    if self.page_size_query_param:                                      
        try:                                                            
            return _positive_int(                                       
                request.query_params[self.page_size_query_param],       
                strict=True,                                            
                cutoff=self.max_page_size                               
            )                                                           
        except (KeyError, ValueError):                                  
            pass                                                        
                                                                        
+   if self.page_size is None:                                          
+       return api_settings.PAGE_SIZE                                   
                                                                        
    return self.page_size                                               

This works. As far as I can see, this is not the proper solution, as it only focuses on page_size and forgets any other DRF setting. I may override this behaviour in my code, but I believe that I may have missed the solution, as it was discussed and ¿solved? in #2466

Thanks for your help.

[awbacker]

Trying to implement a custom throttle class and running into this, just like OP. I think different settings values than production is necessary though, especially to test infrastructure additions like throttling or pagination on a full request. I don't want to rate-limit my unit tests :D

I suppose, since these are declared in the class and not set at init time, they can't be replaced by a settings reload. For list [] settings I suppose the values could be replaced in the list, but this would fail for strings/ints.

• Add a 'lazy' value wrapper that is used in all the base classes
• Set the values of things like throttle_classes in __init__()
• Only read when needed (as mentioned)

Hope this can get fixed at some time, since its making testing a lot of things nearly impossible.

I've worked around this with a decorator, but this is only for throttling:

@contextmanager
def set_throttles(self, view, **rates):
    with patch.object(view, 'throttle_classes', [MyThrottleClass]):
        # set the throttle rates.  you only need to supply the rates that will be
        # used in this particular case
        with patch.object(MyThrottleClass, 'THROTTLE_RATES', rates):
            yield

class Test(TestCase):
    def test_throttle_on_view(self):
        with self.set_throttles(MyLookupView, myscope="2/m"):
            args = {"args": "not-real"}
            self.api_lookup.post(args, status=404)
            self.api_lookup.post(args, status=404)
            self.api_lookup.post(args, status=429)

[raywtham]

I'm also facing this issue. I'm using Django-Q and need to set 'sync':True in order to run async tasks synchronously during tests.
override_settings does not have the intended effect, I need to change the settings file itself.
Is there any workaround?

[abodacs]

@raywtham Use another setting file have 'sync':True to run your tests
hope this link will help
https://simpleisbetterthancomplex.com/tips/2017/07/03/django-tip-20-working-with-multiple-settings-modules.html

[tomage]

Just wanted to throw in my support for this.

I, too, was having issues because if, for example, rest_framework.schemas is imported too "early" (i.e. before Django would load up the settings), my process would fail with ImproperlyConfigured exception, because a Django setting variable is used in the definition of a top-level function signature, and the settings hadn't been loaded up at that point.

I can work my way around it, but it's not optimal.

Also, using settings in this way is discouraged by Django, as per https://docs.djangoproject.com/en/dev/internals/contributing/writing-code/coding-style/#use-of-django-conf-settings:

[...example importing and using settings in top-level...] That means that the ability for third parties to import the module at the top level is incompatible with the ability to configure the settings object manually, or makes it very difficult in some circumstances.

I'm circumventing this in my code by moving the import inside a function that's run much later on, instead of importing at top of my file.

[tomage]

Oh - perhaps to clarify - my problem wasn't really even with overriding settings - it's something that totally broke the python process if certain DRF files were loaded in a bit prematurely. So I'd def. see this as a larger problem than overriding settings during testing. Just to throw a bit more weight on the issue.

Was wondering if I should file this as a separate issue, since it manifests very differently, but also wasn't sure if it would just be extra noise.