User accounts

[taneliang]

An issue describing the general approach for user accounts. TL;DR: Let's just use Firebase Cloud Firestore and pay a few dollars every peak period.

Feel free to edit and add on to this, or shoot this down.

Benefits

• No more hacky QR code syncing
• Just about everyone uses multiple devices - no more timetables stuck on your home computer
• Other developers can integrate better with student timetables without resorting to hacky methods like using the share URL. This will allow an ecosystem to be built around NUSMods, e.g. timetable planners, common free time finders, etc.

Features

Core

• A sync server should be implemented just to handle sync. It should not be integrated with the API (data) server as they don't need to share data, and this will allow us to work on them separately.
• The server should store only preferences and the user's timetable, along with their names and optionally profile pics.
• The server should sync in real-time so that we won't have inconsistent states.
• The server should be ready for multiple timetables per user
• The sync service should be able to continue operating for years without maintainance

Optional

• The server can support normal REST/GraphQL endpoints for third-party developers and applications which don't require real-time syncing.
• API keys for third-party developers for access control.
• Users should be able to modify their timetable when offline, and have those changes synced.
• Sync clashes should be handled automatically

Implementation

Authentication

I think it'll be easiest if we outsource authentication so that we won't have to deal with all the complexities.

If we use Firebase, we can just use their authentication feature as well. A prebuilt UI component and official React component are available. ezpz.

Alternatively, Auth0 seems like a good choice, and they have an Open Source Program which lets us use it for free. They host email/password accounts, support multiple authentication providers like FB and Google, and provide a unique user ID per user. Authentication can be done client-side using their login UI component. The client will then send an access_token JWT to our server when performing requests. This may be vulnerable to replay attacks though.

Good ideas

• Firebase: basically our ideal solution, but we need to pay a bit. We'll probably be only using Firestore and Authentication, which can be cheap/free.

  • Pro: Although I think we'll still cross the free limits for Firestore during peak periods, I don't think we'll pay much as we'll just be charged for what we use beyond the free limits. Here are the Firestore pricing details and the free quotas. The 100 concurrent users limit that we feared also only applies to the Realtime Database, which we won't be using.
  • Pro: Auth is included.
  • We can store a subset of the apps's Redux store in Firestore as a document, with one document per user.
  • We can use Cloud Functions to create and delete a user's document when a user is created and deleted. This guarantees that a user always has a document so that we won't need to pay for security rules to ensure this when updating the db. We also need to prevent the user from adding and deleting documents using security rules.
  • Pro: Data access can be restricted using security rules. These rules may access other documents in the db and we'll be charged for those reads, but I don't think we'll need to do that.
  • Pro: Data can be also be validated server-side using those security rules.
  • Pro: Data is provided in real time.
  • Pro: npm package provided for easy client integration. We can put this in a Redux middleware to minimize the changes to our existing code and to loosely couple Firebase to our app.
  • Con: I'm not sure if we'll be charged for db access if we get spammed with requests. We can whitelist domains to mitigate this risk.
  • Con: I'm not sure if we automatically have an API for third party devs. We may still need to write a server for this.

• Parse: Using Parse Server and Parse SDK for JS, we can roll our own Firebase alternative. Real-time updating is possible with Parse LiveQuery on the server and on the client. We do need to host this ourselves though, and Parse development seems consistent but slow.

Given that Firebase does almost everything we want for a small fee, I think it's the best option that'll save us the time and effort that we'll need to set up or implement our own solution. Parse seems like a good backup for Firebase if it ever shuts down or otherwise becomes unusable. But here are some other alternatives anyway:

• GraphQL subscriptions: Using Apollo server and client, we can implement both the real-time and non-real-time APIs at one shot. However, the Apollo client might not support multiple GraphQL schemas and endpoints, which may be an issue as we move to a GraphQL data server. I'm also not sure if it has great offline and merge conflict resolution.
• Server-side Redux: We can implement a simple sync server by broadcasting a users' client-side Redux actions to all their connected devices using Socket.io. Add server-side state storage and a REST/GraphQL API and we basically have the server that we want. Offline mode might be a problem though, although we might be able to get away with just recording pending actions on the client and blasting them to the server when the network is restored.

Bad ideas

• FeathersJS: seems like a nice Firebase alternative, but its major issue is its lack of offline support. Also, it may not have the most vibrant community, which may pose issues in the long run.
• Google Drive: store data in a random document in users' accounts. Not real-time and rather hacky, but can be done completely client side. This also will not allow third party devs to integrate with NUSMods.
• RethinkDB: basically Firebase's database feature, but open-source and self-hosted. We could connect to it directly from the client, but as it does not have custom access control built in, anyone will be able to read all data from the server. Probably not a good idea.
• Gun: a P2P solution. It's a real-time, offline-first design with conflict resolution built in. We can write a simple server to be both facilitate peer discovery and be an always-on source of truth. However, everyone can read and write all data on the server. We'll need to encrypt data before pushing to the server. It's under heavy development so this may be too unstable.

Potential issues

• Authentication and data leaks
• We need to find a way to manage zombie accounts and unused data
• The server must validate data to prevent nonsense from being stored

[ZhangYiJiang]

For authentication, have we considered IVLE Oauth? Through that we may be able to get some additional data / integration through their API.

[taneliang]

That's possible, but as they're changing the IVLE API it may not be the best idea to integrate that now. Also, we could use NUS OpenID, but I don't think that'll give us access to the IVLE LAPI. Either way, we'll need to write a custom auth server to hook those up with Firebase, which should be simple.

I don't think there'll be any benefit now though. Since Firebase supports linking multiple auth providers to one account, we can add an IVLE login once the new APIs are up.

[ZhangYiJiang]

Hmmm, Firebase is preeetty big (115.8kb minified + gzipped) https://bundlephobia.com/result?p=firebase@4.9.1 Wonder if we can tree shake it

[taneliang]

Interesting site. Does BundlePhobia include a package's dependencies in its sizes?

I don't think we need the firebase package actually. It includes a lot of stuff that we're not using, and it doesn't include Firestore.

• @firebase/app: 6.6kB (supports tree shaking)
• @firebase/firestore: 73.6kB (supports tree shaking)
• @firebase/auth: 43.6kB

Looks like it's still about the same size if we use these packages. If we go the Firebase route I'm not sure if there's any alternative though. And there aren't any good alternatives to Firebase.

Also the UI stuff:

• firebaseui: 104.8kB. This thing has firebase as a dependency though.
• react-firebaseui: 6.8kB

Yeah, that is preeetty big. We could try them out, and if we really can't shrink these then we can just roll our own UI I guess.

Also I looked up Auth0's lock component and it's 197.9kB.

[akshayeshenoi]

Hi guys, I love what you have done here with NUSMods. BIG thanks!

Just curious to know what the status of this feature is. I feel outsourcing to NUS OpenID makes most sense. I'm new to Uni so I'm not sure how complex that could get but I have worked with OAuth systems before.

If any progress has been made on #1104, the auth part obviously could go in there. I see you're using netlify to serve the website so this server could sit been the frontend and netlify. Tons of options for persistence.

Let me know how I could help and thanks again for all the great work!