Skip to content

mentors.debian.net new schema

Baptiste Beauplat requested to merge lyknode/udd:mentors-new-schema into master

Since the django/python3 update on mentors.debian.net, the database schema has changed.

Instead of fetching data directly from mentors' database, a new API be can used to retrieve all necessary info.

This MR is an attempt at updating UDD (its schema for mentors, mentors' gatherer and the DMD) in order to use this new data.

A couple of notes:

  • That the first time I ever wrote ruby. So it has most certainly some bug/stuff missing. Meaning I don't have an extensive knowledge over what I did. (But I can fix anything as long as the error is pointed out and searchable on google :))
  • I tested the schema and the gatherer. The DMD was modified without testing.
  • Mentors' data on UDD has been significantly reduced, but is sufficient for what the DMD does. I'm only fetching the uploads at the moment. While the package name is hinted in the results, we could also fetch the entire package list and split the data into two tables. Simplicity over data completeness.
  • I have no idea how often the data is fetch but mentors currently allow fetching the API every 15 minutes. Anything below that will trigger throttling.
  • This MR references #966297

Looking forward to your comments,

Merge request reports