This project migrates Zulip's backend endpoints from using the has_request_variables decorator to use the typed_endpoint decorator. This new decorator leverages Pydantic v2 to provide type checking and data validation for the parameters passed to the various endpoints, thus making the process of creating new endpoints with complex data parameters easier. Migrating the existing endpoints makes it possible for all the endpoints to have a uniform way of performing data validation and enables them to use Pydantic's BaseModel and validators to provide additional validation for more complex data parameters.
-
I started by migrating some of the more frequently used endpoints (endpoints in
users.pyanduser_topics.py) during the community bonding period. -
I migrated the most used endpoint
message_fetch, which as the name suggest, is used to fetch messages that meet the criteria passed in the parameters. Following that, I migrated some of simpler endpoint (simpler parameters with less validation). -
I proceeded to migrate the endpoints in
streams.pywhich required quite a few changes to the way the parameters were being vallidated due to the complexity if the parameters being accepted by the endpoints in this file. -
I proceeded to migrate the remaining endpoints. In total, I have migrated 150+ endpoints to the
typed_endpointdecorator. -
The common valdiators used for these endpoints are located in typed_endpoint_validators.py.
-
I wrote additional test cases to provide more robust testing of the new validation and modified existing test cases to reflect the new error messages for parameters failing vallidaiton.
-
I modified the tutorial documentation to accurately demonstrate how to create new endpoints and validate their parameters.
- Users typed endpoint
- message_fetch: Convert to typed_endpoint.
- Migrate to typed endpoint
- custom_profile: Convert to typed_endpoint.
- Corporate typed endpoint migration
- streams: Convert to typed_enpoint.
- Integrations-typed-endpoint
- Realm-typed-endpoint
- Zerver-typed-endpoint-migrate
- endpoints: Migrate to typed_endpoint.
- Typed-endpoint-docs
- message_send: Convert to typed endpoint.
- decorators: Migrate to typed_endpoint.
- scheduled_messages: Migrate to typed_endpoint.
- analytics: Migrate to @typed_endpoint.
- Has-request-variables Removal
Initially, understanding exactly how Pydantic handles different data types and parses Json, along with its different validators was tricky. However, after migrating a few endpoints, I developed a good understanding of it, making the process of migrating subsequent endpoints much quicker and easier.
Creating BaseModels and combining them with multiple validators for the more complicated parameters was challenging, especially when ensuring that the intended validation and data conversion was being performed accurately for each of their keys.
The migration has been completed along with the removal of the has_request_variables decorator and the REQ function.
Incrementally migrating 150+ endpoints over the summer has given me a much deeper understanding of the Zulip backend. I have learnt a lot from this project and the additional issues I took up me during this summer at zulip, out of these, the one that stands out the most is my improved ability to explain my changes by writing more comprehensible commit messages and better documenting my code, making me a better programmer overall. The immense sense of satisfaction I felt everytime a freshly migrated set of endpoints passes test cases cannot be put into words. I would like to thank my buddy @Pieter CK and everyone else who helped review my code, especially, @Tim Abbot and @Zixuan James Li.