I love writing a startling heading.
Jugaad is the gutsy art of overcoming harsh constraints to solve a problem using limited resources.– Harvard Business Review
My constraints are a lack of a formal degree in Computer Science. My most limited resource is time. My problem is a bunch of unknown unknowns in software.
I’ve been a bit misguided in treating my ignorance in software architecture independently from my deep insecurity in my programming skills. System Design Thinking is something that lets me take a simultaneous stab at programming and software architecture. I’m totally punching above my weight by setting this gargantuan goal of understanding both.
My current understanding is hardly three questions deep.
- What are the business requirements? What are the minimum viable features in the product?
- How do I represent the structural and behavioural features of this product through a class diagram? (Object-oriented concepts)
- What service APIs will be used for this product? Third-party and internal services? What will call it and how?
- What is the availability of the system? If a host or a data centre goes down, what will be the recourse?
- What is the latency and how can the performance be improved? (Caching, Use CDN, HTTP/2, Reduce JS files etc.)
- Will the system scale to n or more users?
- How to securely store data in the database without losing it? Backups?
- What is the level of security and privacy to be provided?
- How can we make the system cost-effective?
Btw, expert developers reading this might find it either lacking or irrelevant to the subject. After all, what is a part of system design is entirely dependent on whether you think it is important.
To answer these questions I will require sufficient knowledge on the following non-exhaustive list of topics I’ve been gathering through marathon Stackoverflow browsing. I will be adding links to each of these topics as and when I find good ones on the web. Please do suggest links that you enjoyed or benefitted from reading in the comments section.
I hope to teach myself these topics and tools.
DATABASE CONCEPTS
Vertical vs horizontal scaling
CAP theorem
ACID vs BASE
Partitioning/ Sharding
Consistent Hashing
Optimistic vs Pessimistic locking
Strong vs eventual consistency
Relational Databases vs Non-relational SQL
Types of NoSQL
Key value
Wide column
Document-based
Graph-based
MEMORY CONCEPTS
Caching
Data center/ Racks/ Hosts
CPU/ Memory/ Hard drives/ Network bandwidth
Random vs sequential read/ writes to disk
NETWORK CONCEPTS
HTTP vs http2 vs WebSocket
TCP/ IP model
ipv4 vs ipv6
TCP vs UDP
Domain Name Server lookup
Load Balancer L4 & L7
Content Delivery Networks (CDNs) & Edges
SECURITY CONCEPTS
HTTP & TLS
Public key infrastructure and certificate authority(CA)
Symmetric vs asymmetric encryption
OTHER CONCEPTS
Paxos
Leader election
Design patterns and Object-oriented design
Virtual machines and containers
Publisher Subscriber over a queue architecture
MapReduce for distributed and parallel processing of big data
Multithreading, locks, synchronization, concurrency, compare-and-swap
Data Structures & Algorithms relevant to the system
Probabilistic Data Structures: Bloom filters and Count-Min sketch for big data
Tools
Cassandra
MongoDB/Couchbase
Mysql
Memcached
Redis
Zookeeper
Kafka
NGINX
HAProxy
Solr, Elastic search
Amazon S3
Docker, Kubernetes, Mesos
Hadoop/Spark and HDFS
Edit: An awesome colleague pointed me to this link: https://github.com/donnemartin/system-design-primer
Observation Book 