tech/README.md - how to document your repo
TLDR
Use github for documentation. For multiple services, use separate repo. Tie your tech doc to product. Create readme.md in your repo and add there:
- Description
- Usage
- Dependencies
- API / DB schema
Types of docs and what this recommendation is not about
Documentation can be of different type, depending on its purpose and end-user:
- specification document with requirements prior to development
- design document incorporating multiple services, UI, hardware components
- RFC (request for comments) as reference for a specific protocol or implementation piece
- integration document shared with other vendors
- public API documentation (interactive with swagger, postman, apidog etc.)
- public product guides (screeshots, videos) and integration docs (how-tos, usage, code snippets)
This recommendation discusses documentation systems that is needed for web-app as a product for internal engineers or for open source repos used by external engineers.
Single service README structure
These are all optional, pick ones that are relevant to you. "Priority" column here is just to understand which ones are more important.
| Naming | Details | Priority |
|---|---|---|
| Marketing - Badges, logo, ToC | Useful image badges highlighting test coverage, security vulnerabilities, license, platform etc. | 2 |
| Overview | Problem statement and purpose of the service. Screenshot of the UI, if applicable | 1 |
| Prerequisites | What are the expectations for the client system. Hardware, OS, Docker, Node version | 3 |
| Installation | How to run locally. Installing dependencies, docker, shell scripts, manual actions | 3 |
| URLs | If this is BE service, which URL and port is it available on and on which environment. A table is nice to have. | 3 |
| Usage | How do I call this CLI, service. For UI which pages, feature flags, user settings, payment tiers do I need. | 2 |
| Stakeholders | Who uses this service directly, who is affected (teams, roles, clients, third-party developers..), who is the owner. | 3 |
| Roadmap | What is the future of this service. What are the plans. How are new features discussed. How it evolve (roadmap link), will service get merged or be deprecated | 3 |
| TEX video(s) | Explain how service works (if it was transferred from another team). What to look after. ~1h video usually. | 4 |
| Product requirements | How does service affect product experience? Link to a bigger design document explaining full architecture. | 1 |
| Use cases / FAQ | Stakeholders, actions, multiplicity. Example - describe typical scenarios that developers follow (adding some new code, extending a service etc) | 2 |
| Development | • How to build (global tools if needed) • How to run in hot reload mode • How to debug in IDE / docker, profile, benchmark • How initial DB is set up. Where are migrations & fixtures | 3 |
| Scripts | - useful npm, just, make or bash scripts that are available | 3 |
| Testing & Quality | • How to run unit tests (how to make snapshots) • How to run functional tests (how they work) • Badges (sonar, codeship, snyk) | 3 |
| Deployment / Publishing | • Deployment links (jenkins) • Monitoring links ◦ Grafana dashboard ◦ Graylog • Prometheus endpoints for monitoring? What metrics are important/tracked? • Deployment diagram. Useful in case of complex server type or non-standard region setup (backoffice, marketplace) • Multi-DC diagram / region specific logic | 3 |
| Monitoring / Logging | - List exposed prometheus metrics - link to grafana/newrelic dashboards - link to graylog, opensearch or sentry logs for quick access | 3 |
| Configuration | - Where is config read from. Vault, Env vars, Consul, Filesystem, DB? - List of env vars. Default values. What do they do. | 2 |
| DB schema | • Use mermaid-js • Link to schemas / diagram (if service has any DB at all). Lucidchart can automatically generate it based on DQL dump | 1 |
| Events | Generated for external message queues, tracking or analytics engines • Fastify event schema (typescript) • Links/accounts for google analytics, newrelic, fullstory • Schema, fields, type. Link to tests | 2 |
| Tech stack | List if stack is non-standard. Include versions • Language & extensions • Framework • Third-party product (cms, forum etc.) | 3 |
| Architecture diagram | Can include - what service depends on (as network requests) - what does its known to affect via events - use mermaidjs diagram. Diagram nodes can link to other repos. • services ◦ external SaaS services & APIs ◦ CDNs • database servers • message queues • background (cron) jobs should include start time, frequency and name | 1 |
| Libraries | What code it depends on as its a major security factor Mention third-party dependencies if they are not standard for your stack (package.json, composer.json, Gopkg.toml) Make sure that license allows commercial use | 3 |
| Entities & relationships | Concepts and terminology, their one-to-many relationships. This can be seen as more general version of DB schema diagram, but can include entities that are not only in DB, but also in external services, filesystem, caches etc. | 3 |
| API | - type of API used (REST/GraphQL, GRPC). Where/how is it generated/exposed. • Plain list of endpoints & methods that are used ◦ in markdown, you can use collapsible blocks • Draw a diagram Useful if service has 3+ endpoints, has non-restful interface. Can highlight what endpoint calls what sub-service / db • link to postman collections • Include input payload & response examples (if swagger is not used) • Compose swagger.json manually • Compose swagger.json on the fly based on code annotations • Serve swagger.json via API based on live code & schemas • Exposed public function reference (for libraries) | 2 |
| Page navigation diagrams | Useful in case of web-app. Useful to show how user can navigate across UI | 3 |
| Class diagrams | - Useful to show deeper waterfall or inheritance structure or highlight specific functionality - Sub-systems. How important custom code works (Caching, DB query builder, Localization, Event processing, Logging, Email generation & delivery, Search, File handling, Encryption). Typically these are candidates for extraction to a separate library or a service | 3 |
| Business rules / State diagram | These are typically complex if-else scenarios important to how product behaves | 3 |
| Sequence diagrams | Data flow diagram with time directed from top to bottom. Useful logic is complex with multiple involved services, to show response/error code importance | 3 |
| Contribution | What needs to be done before PR (linters, codestyle, version bumping, changelog, drafting a release for a library) How should pull-request review happen (slack notifications) Who can deploy at what time. List of notable contributors (open source hall of fame) | 2 |
| License / Citation | Mention if custom license is used. For research paper repos, include how this should be cited. | 3 |
Project architecture
Multiple services are usually grouped by their role for the product. Project is a big feature group of web-app or a separate product / stand-alone portal
Diagrams
Diagrams are visually pleasing, but maintaining them is harder. They are also not as searchable, so keep them to minimum
You can use:
- mermaid-js recommended as it is built-in into github, has IDE plugins, thus does not need any image re-upload or re-generation
- excalidraw
- lucidcharts (ask internal IT for access)
- draw.io
- websequencediagrams
Or other SaaS service that is editable later. Should not need explicit permission. Link is needed in each case. Should not need custom software In each component/class/queue description put in a name that can be searchable through the code or in consul. If you use lucidchart, try to include a legend to explain what certain shapes mean. Typically, prefer using UML 2.0 syntax
More theory
Wow, you reached this section. You must be an aspiring librarian wizard
Documentation role
Documentation is needed to answer most important question - Why (service is needed)?With technical documentation, question often transforms more into How (does it work)?
- explain purpose, reasons, limitations of having this service
- explain what is the role and roadmap of the service in a product
- efficiently share the knowledge of most important parts across tribes
- formalize requirements to have common expectations of what are service limitations
- synchronise terminology that service uses and exposes with other parties
- improve quality by having automated tests acting as living documentation that proves that service works as expected
- act as map, linking to implementation details
Requirements
To answer How (service works), you can list functional requirements.
Architecture is expressed by constraints. To answer Why service was written the way it is, mention non-functional requirements.For the product its often: Performance, Reliability, Security, Usability, Configurability, Availability, ScalabilityFor development its often: Maintainability, Localisation, Compatibility, Reusability, Extensibility, Portability, Interoperability, Accessibility
See https://confluence.atlassian.com/doc/blog/2015/08/how-to-document-product-requirements-in-confluence
Language style
Documentation should be
- Simple. Avoid slang and abbreviations. Provide examples for complex calculations
- Incomplete. Documentation cannot cover everything, that is what code is for. Instead its an overview of significant, high-level things worthy of notice. Think of google maps with its zoom levels and layering.
- Consistent. Content should not contradict itself and use same terminology
- Precise. Things that you should avoid:
- generalizations (all, everywhere, never, any, each)
- vague verbs (optimize, improve, support, handle)
- vague adjectives (easy, simple, seamless)
- vague references (like in product X)
- not countable values (few, several, many, some)
- open-ended statements (etc., where appropriate, sufficient, optionally, including but not limited to)
- recursive and/or double meaning sentenses (see last point)
Keeping documentation in sync
Documentation tends to get out of sync because developers don't use documentation, so they don't update it & rely on code as single source of truth.
Suggested approach is to
- Keep service docs in github
- Have diagrams editable by any developer (without explicit permission request)
- Update docs with every github PR
- include diagrams included in PR description