NUBIA UNDER THE HOOD

Get to know more about how Nubia works under the hood, navigating the site, our thought process and tools used

The tech Stuff

Nubia: what's going on?

As of now, Nubia has 2 major systems: The Client & Server. The Server does the story generation while the client side handles all of the rendering and display of articles. The Server was built using Express Js, a framework for serving nodeJs apps. The API requests are captured by controllers and then these controllers call the services required based on the specification of the reqeust.

Before a route can be valid for use, a series of things have to be put in place. These things include

Creation of Story template
Adding of csv data source
Adding of service for new story
Parsing of csv Data to JSON
Adding of controller for new story
Connecting the whole system in the sequence: Template → Service → Controller → Route

NUBIA
Client → NextJs/React, Tailwind, SWR, Zustand
Server → ExpressJs, Rosaenlg, csv-parser
Major server language is Typescript

IntroducingFollowing the sequence

After successful wiring up of the various components, the route is ready to be queried by a client and used to generate stories.

Creating the template

For templating, Nubia uses Rosaenlg which is an NLG open-source tool built ontop of Pug (Actually, it's pug on steroids). Rosaenlg adds additional features to pug, making able to perform tasks pug normally would not ba able to achieve. This can be a little bit tricky as there is no syntax highling for rosae and a lot of the commands would be flagged as in-appropriate by the IDE or syntax formating plugins.

faac-template — An example rosaenlg template very similar to pug

Adding CSV Data source

At the moment, the data files are added to the repo under the data folder. This data is then run through an utility function that parses the data to Javascript objects. Very soon, data storage would be moved into a cloud file storage like AWS S3, Google cloud file storage, Azure or whatever cloud platform is deemed worthy at the time, then the urls would be stored in a database. The data parsing utility would also be re-written to accommodate cvs from urls

Creating a service for the story

After creation of the temoplate and adding of data, a service should be created to house the logic of the story creation: parsing the data, calling the template engine, storing data to database and all other major logic.

faac-service-image — Example service creating FAAC stories

Parsing of CSV data to JSON

There is a data parsing utility provided in the code repo that supports csv and takes in 3 parameters which are

The data to be parsed
A callback function that gets called automatically by the parser and passes each entry on the data as a argument to the callback.
A final call back function that passes the final parsed result as argument and calls the callback upon parse completion

nubia-parser-utility — Parser utility function

Create the Controller for the Story

After completion of service, a controller is created to handle incoming requests, assign classnames and call the appropriate services.

Wiring it all to a route

After the controller is ready and sends a valid response object to the user, a route would be created specifying the story end-points. The typical naming convention for this is host.com/story-name and the route for getting the corresponding data is host.com/story-name/data