[RMP] Centralize Documentation into the Merlin/Merlin repo

[bschifferer]

Problem:

The feedback of our repository is that it is easy to follow:

• Every repository has its own documentations. Users have to switch back and forth between documentations
• There is no single entry point for users where to start and read something end-to-end

Some examples:

• TensorFlow: https://www.tensorflow.org/tutorials ; https://www.tensorflow.org/guide
• Ray https://docs.ray.io/en/latest/

Goal:

Provide a better user experience for our documentations

High Level Decision:

We need to decide:

• How should (new) users discover Merlin?
• Do we have one centralized documentation vs. documentations per repository?
  • [Task] Centralize API Documentation in Merlin  #1053
• • • If centralised - what does it mean? Do we move all API docstring documentation into one repository?

ADR: https://docs.google.com/document/d/1FmK55tAo0wy0tHUe3eE8xqZKGDYSvy0tvq8Y036854A/edit#heading=h.dspp0kxpgck1

Some Options:

• Move the docs to a single main Merlin Repo
• Rewrite Introduction Page: https://docs.google.com/document/d/1jp6wGZb6x9qVXR5axy0_SSeZ1nI3vLcfpvGH_A0vc-w/edit
• Add additional pages for Recommender System Guide: https://nvidia-merlin.github.io/Merlin/main/guide/recommender_system_guide.html
• • Recommender Systems - Systems - Deployment/Multi-Stage
• • Advanced Topics - Multi-Task Learning, Pre-Trained Embeddings, Negative Sampling, etc.
• • Scaling - Data Parallel Training / Model Parallel Training ; Distributed Embeddings
• Installation Guide: [WIP] Adding Installation Description #811
• Docstring coverage (and reviewing if docstrings are correct?)
• Integrating the examples into Documentation
• Move examples that are split across repos (Systems, Models) into Merlin repo

[bschifferer]

I started a draft for a Merlin Decision Process:
https://docs.google.com/document/d/1FmK55tAo0wy0tHUe3eE8xqZKGDYSvy0tvq8Y036854A/edit#

[oliverholworthy]

Started a couple of things that help toward this goal of improving the docs

• Settting up the stable branch as the default branch for docs sites [INF] Update default references to main branch to latest release #866
• Updating the theme and adding a sidebar with links to other projects in the merlin ecosystem Update Docs Theme to use sphinx-book-theme core#330