OpenWISP REST API: GSoC’21 Project Report

10 min readAug 20, 2021

New Admin Interface and REST API endpoints in action. — New OpenWISP Admin UI and REST API endpoints.

This summer was the most productive of all I had ever had, for the past few months, I have been contributing to OpenWISP, an Open Source organization selected for Google Summer of Code 2021 program. Mostly my work has been centered around openwisp-controller, openwisp-users & openwisp-network-topology modules. I wrote more than 5000+ lines of code and opened 21+ PR’s, under the mentorship of Federico Capoano, Ajay Tripathi, and Oliver Kraitschy. This article sums up all my work done during the GSoC period.

Source: https://openwisp.org/

OpenWISP is a software platform designed to ease and automate the management of networks, with a special focus on wireless networks, mainly used in public wifi, mesh networks, community networks, and IoT scenarios.
— OpenWISP.org

Aim of the Project:

The main objective of this project was to fulfill the most important and much-needed missing REST API endpoints for some of the Django models of the oldest OpenWISP modules. The inclusion of this feature will not only add more usability to these modules but will also help in the faster development of other new features, and also save time during development. It will also enable communication between the application and the database which will ease the integration of OpenWISP features with other software or applications.

Setting up the Kanban Board:

“A goal without a plan is just a wish.”
-Antoine de Saint-Exupéry

So, after the results were announced, without wasting much time I was asked to create all the necessary issues for the features, I am going to add and move them to the Kanban project board to keep a track of the project todo’s and the priorities. The OpenWISP REST API kanban project board can be found here. Which will contain more than 35+ card’s including issues and PR’s in the Done column after the program ends. Also, I had to send a weekly progress report of my project status to a public mailing list, which was specially created for the GSoC21 selected students. The thread for my project can be found at Weekly Progress Report — OpenWISP REST API[GSoC21].

Tech Stacks Worked On: Python | Django | Django Rest Framework

Project Details:

So, let’s now dive into the project in-depth and understand what we will achieve by the completion of this project. My entire work was concentrated mainly on these two frameworks:- Django, django-rest-framework. Since all the modules of OpenWISP are built majorly with Python, and Django Therefore It was Pre-decided that all the API endpoints be built with django-rest-framework which provides a default web browsable API, awesome authentication, and permission classes, also due to its generic views it becomes huge usability wins for the developers.

This project can be broadly divided into three sections, based on the modules of OpenWISP:-

openwisp-controller
- Config App
- PKI App
- Connection App
- Geo App
openwisp-users
openwisp-network-topology

Started the work with the openwisp-controller module, since it demanded much more work as it contained four Django apps in it compared to other modules.

What is openwisp-controller?

OpenWISP Controller is a configuration manager that allows to automate several networking tasks like adoption, provisioning, management VPN configuration, X509 certificates automatic generation, revocation of x509 certificates and a lot more features.
— openwisp-controller(docs)

Config App: This app allows us to configure devices in a network with the help of configurable templates, containing the configuration. which can be further added or removed to devices to reduce redundancy. Also, VPN can be related to a template, and automatically provision VPN tunnels with unique x509 certificates. Built the API endpoints to allow CRUD operation for the Template , Device , and Vpn models.

PR #386: [feature] REST API for main controller features #379

This was my first major work, which acted as training for me. I started the work as soon as the project idea was announced. I had to follow a somewhat similar pattern for all the other apps and modules I would be working on. I learned what, how, and why about most of the major applications of django-rest-framework, django, and how it will be integrated with the other app. We had to tackle major bugs and issues that had to be resolved first before we could move towards other modules and apps.

PKI App: The PKI app is based on django-x509, it allows to create, import, and view x509 CA’s and Certificates directly from the administration dashboard. This app contained two models:- Ca and Cert . So, had to introduce the API endpoints for both the models which should allow performing CRUD operation on it. The code and discussions can be viewed by the PR link given below.

PR #455: [api] Rest API for PKI app #455

Connection App: This app enables the controller to instantiate connections to the devices to perform push operations:- Sending configuration updates, Executing shell commands, Perform firmware upgrades via the additional firmware upgrade module. So, this app contained two models: Credential and DeviceConnection . Which didn’t take much time in introducing the required endpoints.

PR #497: [api] REST API for the models of connection app #464 #497

Now, till here all the logic's and implementations were simple and easy and didn’t require a much deeper dive into django-rest-framework’s implementation. So next moved to Geo App which required a bit more time to understand how to implement, and the special cases which needed to be handled.

Geo App: The geographic app is based on django-loci and allows to define the geographic coordinates of the devices, as well as their indoor coordinates on floorplan images. The app contained three models as Location , FloorPlan and DeviceLocation . Where the model DeviceLocation had to be handled with proper logic, for the devices containing Location objects, so had to understand the concepts of proper RESTfull API endpoint architecture, also had to handle requests in a slightly different manner. The work for the GEO app is in progress and in the review/testing stage at the time of penning down this blog.

PR #494: [api] REST API for the models of GEO App #463 #494

With this, we have completed the work of openwisp-controller module, Now we shall move on to openwisp-network-topology module. So,

What is openwisp-network-topology?

OpenWISP Network Topology is a network topology collector and visualizer web application and API, it allows to collect network topology data from different networking software (dynamic mesh routing protocols, OpenVPN), store it, visualize it, edit its details, it also provides hooks (a.k.a Django signals) to execute code when the status of a link changes.
— openwisp-network-topology(docs)

I was required to make the API endpoints for all the CRUD operations for these three models:- Topology , Node , and Link . Working on this module came with its own challenges, I had no experience of working with NetJSON before, so took some time to understand it and its implementation. Finally, after deep-diving into it making concepts clear about its representation and need I was able to get things done on time. The work is in the review/testing phase. A glimpse of the endpoints can be seen in the below GIF.

PR #113: [api] REST API for openwisp-network-topology module#109 #113

OpenWISP Network Topology REST API Endpoints

Let’s now have a look at the work accomplished in the user’s module.

What is openwisp-users?

openwisp-users module contains the implementation of user management and multi-tenancy for OpenWISP.
— openwisp-users(docs)

In, this module I had to mainly introduced endpoints for the User , Group , Organization , and EmailAddress model, apart from this an extra endpoint to change the password of users, was also introduced. The most challenging part which I found while working on this module, was the skyrocketing of the number of queries hitting the database. So, to optimize the number of queries, tried to keep the number of queries under check, and used some of the well-known query optimization tool provided by django i.e., select_related() and prefetch_related()in django wherever it was necessary, But I still believe that the number of queries can be more optimized.

PR #264: [api] Rest API for openwisp-users module #250 #264

Before Vs After (query optimization [Group Model])

The work of the users module is ready for review/testing at the time of articulating this blog. Now let’s see what are the core features of the REST API endpoints built.

Salient Features of the REST API:

Pagination
- Also known as paging, is the process of dividing a document into discrete pages, either electronic pages or printed pages. All the endpoints introduced are properly paginated, query_params as page=? or page_size=? can be provided to override the default setting of the pagination params.
Authentication
- The endpoints are only available to authenticated users, which increases the security aspects of the endpoints. This was achieved with the help of some of the well-known classes like TokenAuthentication , BasicAuthentication and SessionAuthentication from DRF.
Permissions
- All the API endpoints introduced checks for all the necessary permissions contained by the client to be able to receive or send data to the resources it requests for. Classes from DRF like IsAuthenticated and custom permissions classes from openwisp-users a module like DjangoModelPermission , IsOrganizaitonManager, IsOrganizationMember, IsOrganizaitonOwner helped in achieving this. The permission class DjangoModelPermission got introduced in the GSoC21 period. With the same logic which we are using in OpenWISP in this permission class, we have also opened a PR #8009 to get it merged in the DRF repo.
Multitenancy
- In the default DRF’s browsable API page the relationship fields in this interface show all the relationships, without filtering the queryset based on the organization to the user has access to, which breaks multi-tenancy. So, to tackle this problem the mixin classes FilterByParentmanaged, FilterByOrganizationManaged and many such classes were taken into effect from the openwisp-users module. Also to filter serializer fields, classes like FilterSerializerByOrgManaged etc. was inherited in the serializers.
Documentation
- All of the three modules I worked on has been properly documented about all the endpoints introduced, with proper usage example and a note wherever necessary in the Readme file of each respective module.
DRF-YASG(Django Rest Framework-Yet another Swagger generator)
- All three modules have the DRF-YASG package included which has been configured in the openwisp-utils module to be used in all the modules of OpenWISP.

Link to some of the other notable PR’s:

Status of the Project

I was able to complete my project as at the day of writing this article, I have opened all the required PR’s that was mentioned in my proposal and met all the deliverable’s, some of the PR’s are yet to be merged and are in the review/testing phase, so will be working on it to fix bugs and resolve change requests if any.

Learnings:

Testing(TDD)
- Before, the GSoC21 period I had little or no knowledge about testing and its importance in software development. All the endpoints introduced during this period had to be properly tested and made sure that the coverage never decreases. So, I had to spend a decent amount of time learning how to write proper tests, and make sure each line of the code is being properly covered.
Documentation
- While working on this project I understood the true importance of good documentation, as the project required in-depth knowledge of the working of DRF, so I couldn't find any article or tutorial which fulfilled my need the only option left was to read the codebase and documentation. Also, we had to document all the features that were added.
Code Quality
- The code quality is important, as it impacts the overall software quality, and quality impacts how safe, secure, and reliable our codebase is. We made sure the code is to the project coding style and convention and follows the best coding practices.
Meeting Deadlines
- Completing a project before its deadline, plays a very important role in a developer's life. Working with OpenWISP really taught me how to manage time and all my mentors made sure that I complete the project on time.

Experiences:

This summer was one of my best, full of learning, development, and networking. I can confidently say myself to be a better developer than what I was three months back. I got a chance to work with advanced developers, which taught me a lot of good coding practices that should be followed in software development and open source. The mentors with whom I have worked were very friendly, helpful, and supportive throughout the program. I’d like to especially thank my mentors Federico Capoano, atb00ker, and Oliver Kraitschy for taking out their precious time to review and test all of my work done in this period. The work was always enjoyable, insightful, and enlightening. OpenWISP has an awesome community members full of enthusiasm who are always ready to support and motivate all the new contributors and members.

What’s Next?

While contributing to OpenWISP for more than a year now, I am now pretty well familiar with the code base and would love to contribute to its newer projects and keep on improving on the project I worked on.