File Uploading at REST API with JSON Parsing into list of route-Points

Custom JSONParser - extending Django-Rest-Framework JSONParser?

Why would you even need to extend JSONParser ? Well... I want to have a list of Points at the end of parsing, not a json-type data.

In that way I can easily manipulate this data and check information on it, i.e. using endorphine-algorithms to find out if badge can be applied to specific route!

How to start working on custom Parser from Djagno-Rest-Framework?

TDD approach of-course! But how? Well I didn't know myself when I started to figure it out, but then I thought to myself- why not check at source-code of Djagno-Rest-Framework - Django-Rest-Framework Parsers Test-case- But unfortunatelly this code does not provided me any valid answers.

Then I've looked into source code of parser and realized that the parse method is the one used. So I've went that way to extend functionality with mine.

I've firstly made an test for checking if only extending JSONParse without changing content and picking parse method will create at the end the same output as input (as a StringIO stream).

Then I've changed tests so they fit my assumptions about behaviour of this custom JSON parser.

Initial tests with source-code:

class RouteJSONParser(JSONParser):
    """ Route JSON Parser - parses JSON file into list of Points"""
    pass


class TestRouteJSONParser(TestCase):
    " Tests for RouteJSONParser class parse method overloaded"

    def setUp(self):
        self.parser = RouteJSONParser()

    def test_parse(self):
        " tests parser "
        input_dict = [
            {
                "lat": "53.126699",
                "lon": "18.072322",
                "time": "2000-01-01T07:47:02Z",
                "ele": 107.0
            },
            {
                "lat": "53.126699",
                "lon": "18.072322",
                "time": "2001-01-01T07:47:02Z",
                "ele": 99.0
            }
        ]
        input_stream = StringIO(json.dumps(input_dict))
        output = self.parser.parse(input_stream)
        assert str(output) == str(input_dict)

Final Source code of RoutesJSONParser with Tests

Tests:

class TestRouteJSONParser(TestCase):
    " Tests for RouteJSONParser class parse method overloaded"

    def setUp(self):
        self.parser = RouteJSONParser()

    def test_parse(self):
        " tests parser "
        input_dict = [
            {
                "lat": "53.126699",
                "lon": "18.072322",
                "time": "2000-01-01T07:47:02Z",
                "ele": 107.0
            },
            {
                "lat": "53.126699",
                "lon": "18.072322",
                "time": "2001-01-01T07:47:02Z",
                "ele": 99.0
            }
        ]
        expected = [
            Point(
                lat="53.126699",
                lon="18.072322",
                time="2000-01-01T07:47:02Z",
                ele=107.0
            ),
            Point(
                lat="53.126699",
                lon="18.072322",
                time="2001-01-01T07:47:02Z",
                ele=99.0
            )
        ]

        input_stream = StringIO(json.dumps(input_dict))
        output = self.parser.parse(input_stream)
        assert str(output) == str(expected)

Source-code - that resides in a test-module for now. I will definitely move that into source-files.

# pylint: disable=too-few-public-methods
class Point(object):
    """ Point object with lat/lon time and ele """

    def __init__(self, lat, lon, time, ele):
        self.lat = lat
        self.lon = lon
        self.time = time
        self.ele = ele

    def __repr__(self):
        return "{},{},{},{}".format(
            self.lat,
            self.lon,
            self.time,
            self.ele,
        )


# pylint: disable=too-few-public-methods
class RouteJSONParser(JSONParser):
    """ Route JSON Parser - parses JSON file into list of Points"""

    def parse(self, stream, media_type=None, parser_context=None):
        json_content = super(RouteJSONParser, self).parse(stream, media_type, parser_context)
        points = []
        for point in json_content:
            points.append(Point(point['lat'], point['lon'], point['time'], point['ele']))
        return points

File uploading via REST-API.

I wanted to follow TDD approach at this, but first I had to check how to create FileUploading solution cause I don't remember when I've done such thing!

So I've broke my "first test than production code", and first created a production code to check how it will look like, So I could find out how I should test it.

Fortunatelly for me Django-Rest-Framework Documentation on that subject.

After struggling for few hours with having still a "Not found" feedback information from source-code in tests, I've passed for now.

I'll figure this out someway, but this will not be today unfortunatelly :(

When checking manually source-code seems to be working, but I'm not sure if it does in all cases because of lack of passing tests... :/

Source-Code:

class FileUploadView(APIView):
    " A view for file upload "
    parser_classes = (JSONParser,)

    def post(self, request, format='json'):
        return Response(request.data)

Tests that is not passing yet:

class TestFileUploadView(APIGeneralTestCase):
    "File Upload tests"
    def setUp(self):
        super(self.__class__, self).setUp()
        self.api = APIRequestFactory().post
        self.view = FileUploadView.as_view()

    def _create_test_file(self, path):
        f = open(path, 'w')
        f.write('test123\n')
        f.close()
        f = open(path, 'rb')
        return {'datafile': f}

    def test_update_user(self):
        " Tests if making api request updates user with new name"

        import os
        dir_path = os.path.dirname(os.path.realpath(__file__))
        file_ = open(dir_path + '/samples/routesample.json', 'rb')

        api_data = {'filename':  file_ }

        self.client.credentials(HTTP_AUTHORIZATION='Token {}'.format(self.token))
        request = APIRequestFactory().post(
            "/api/file/",
            api_data,
            format='json'
        )
        response = self.view(request, format='multipart')
        assert response.status_code == 200

Bugs /Features I've found while working

Bugs:

Non so-far.

Features:

Code commits done for this post:

In branch feature/55-parser-for-routes-in-json-data-format: - Adds tests with RouteJSONParser

In branch feature/46-Endpoint-for-adding-new-routes-should-take-json-content-in-body:

Acknowledgements

StackOverFlow:

Other:

Djagno-Rest-Framework Documentation:

What's next

Contest is almost over. But working on project is not :)

Anndd.....

Monty Python is here again!

Literally Disarmed still does his job

Thanks! See you soon!

Comments

comments powered by Disqus