Toolhub/Progress reports/2020-11-20
Report on activities in the Toolhub project for the week ending 2020-11-20.
No report for week of 2020-11-13 edit
No report was published for the week of 2020-11-13. That week was a short week for Wikimedia Foundation staff.
Toolinfo read API edit
- /api/tools - Paginated, sortable list of all toolinfo records
- /api/tools/{id}/ - toolinfo record
Toolinfo.json crawler and status API edit
A basic web crawler driven by the registered toolinfo.json URLs has been implemented in gerrit:641559. The crawler is currently manually runnable using poetry run ./manage.py crawl in the developer's environment. Each crawl run's outcome is documented in a set of models which are exposed via the API for inspection:
- /api/crawler/runs/ - list of known crawler runs
- /api/crawler/runs/{id}/ - single crawler run
- /api/crawler/runs/{run_id}/urls/ - list of all urls crawled in a run
- /api/crawler/runs/{run_id}/urls/{id}/ - info for a specific URL crawled in a run
Some follow up work is desired to improve the crawler and the stored information, but we have a working system!
Locale switching in Vue frontend edit
Srishti has added preliminary support for locale selection in the Vue frontend app using the vue-i18n plugin and a custom locale switcher UI element. She plans to continue to build on this with RTL testing and improvements in the coming week. We expect to start deeper integration with TWN in early December.
Improved error responses from API edit
We have forked the drf-friendly-errors-2 library to add support for newer versions of Django Rest Framework. To keep things moving, we are currently installing our forked version but we submitted our changes upstream for inclusion in a future release.
Errors from the API will now return results which include error codes to help API consumers detect and respond to API errors with a slightly easier error determination method than message string processing.
Miscellaneous changes edit
Various other changes that landed in the repo in the last 2 weeks:
- Fixed a typo in the GPLv3 header used in Python source files
- Added a Makefile target to run black on the developer's git repo
- "Crawler" prefix removed from names of models in the toolhub.apps.crawler app
- Date sorting fixed in /add-remove-tools screen
- phab:T267261 Upgraded to drf-spectacular 0.11.0 and switched to using it's
@extend_schema_viewdecorator to add localizable descriptions to API endpoints
Wrap up edit
Even with the vacation time in the last 2 weeks, we are making good progress. The implementation of the crawler and toolinfo storage and their related APIs unblocks a lot of UI work. Both the Django and Vue systems are ready to start on the process of integrating with TWN for message translation, which could start happening as early as next week (although the first week of December is more likely). We are also very close to the point of having enough functionality to make demoing with others a useful process.
We are also starting to evaluate our actual progress vs our announced plans for the quarter. Audit logs, TWN integration, and OAuth for API consumers all remain to be done. With only 6 calendar weeks, and functionally 4 working weeks left in the quarter we are cautiously optimistic that all of these remaining components will be started. It is unlikely that they will all be completed however. We should be able to make better estimates in the next 2 weeks about what will and will not be possible in the remaining timebox.