Quartz sync: Jan 25, 2026, 8:08 PM

Author: slyofzero

content/Computer Organization and Architecture/Cache Organization.md

@@ -0,0 +1,189 @@
+>[!SUMMARY] Table of Contents
+>- [[Cache Organization#Locality of Reference|Locality of Reference]]
+>- [[Cache Organization#Working of Cache Memory|Working of Cache Memory]]
+>	- [[Cache Organization#Block|Block]]
+>	- [[Cache Organization#Average Memory Access Time|Average Memory Access Time]]
+>- [[Cache Organization#Types of Cache Access|Types of Cache Access]]
+>	- [[Cache Organization#Simultaneous Access|Simultaneous Access]]
+>	- [[Cache Organization#Hierarchical Access|Hierarchical Access]]
+>	- [[Cache Organization#Memory Access Time when Locality of Reference is used|Memory Access Time when Locality of Reference is used]]
+>- [[Cache Organization#Cache Write or Write Propagation|Cache Write or Write Propagation]]
+>	- [[Cache Organization#Write Through|Write Through]]
+>	- [[Cache Organization#Write Back|Write Back]]
+>	- [[Cache Organization#Write Miss|Write Miss]]
+>		- [[Cache Organization#Write Allocate|Write Allocate]]
+>		- [[Cache Organization#No Write Allocate|No Write Allocate]]
+>- [[Cache Organization#Questions|Questions]]
+# Locality of Reference
+Programs tend to access the same memory location or nearby memory locations within short intervals of time.
+
+Types -
+1. **Temporal Locality -** Recently accessed memory locations are likely to be accessed again.
+2. **Spatial Locality -** Memory locations close to the recently accessed memory are likely to be accessed.
+3. **Sequential Locality -** Access in strictly increasing order of address.
+
+This phenomenon allows for caching a **block of memory** to be efficient. Currently demanded localities are kept in a smaller and faster memory called **cache**.
+# Working of Cache Memory
+![[Pasted image 20260125131640.png]]
+
+Keywords -
+1. Cache Hit - When demanded CPU content is present in Cache.
+2. Cache Miss - When demanded CPU content is absent in Cache.
+3. Hit Ratio $(H)$ - Fraction of times a Cache Hit occurs in all memory references.
+
+$$
+H = \frac{\text{No. of hits}}{\text{No. of memory references}}
+$$
+## Block
+More specifically, a block is a fixed-size contiguous group of memory words transferred between the main memory and cache memory as a single unit.
+
+***Example -*** Whenever a Cache Miss occurs, the CPU retrieves the content from the Main Memory itself. But because locality of reference is known, a **neighbourhood around that content** is brought to the cache to make future memory accesses **more efficient**. This neighbour is a block.
+## Average Memory Access Time
+Both a Cache Miss and a Cache Hit take some time for performing the content transfer. So the average memory access time is -
+
+$$
+\text{Avg. Mem. Access Time} = H*(\text{Time for cache hit}) + (1-H)*(\text{Time for cache miss})
+$$
+
+^a692b1
+# Types of Cache Access
+## Simultaneous Access
+The memory access request is **sent to both** the cache memory as well as the main memory. Hence this is also called a **parallel access**.
+
+![[Pasted image 20260125134951.png]]
+
+The average memory access time is -
+
+$$
+T_{avg} = H*T_{cm} + (1-H)*T_{mm}
+$$
+
+Here $T_{cm}$ is the cache memory access time and $T_{mm}$ is the main memory access time.
+## Hierarchical Access
+The memory access request is sent to the main memory **only when a Cache Miss** occurs. Hence this is also called as **serial access**.
+
+The average memory access time is -
+
+$$
+\begin{aligned}
+T_{avg} &= H*T_{cm} + (1-H)*(T_{cm}+T_{mm}) \\[8pt]
+&= \cancel{H*T_{cm}} + T_{cm}+T_{mm} \cancel{- H*T_{cm}} + H*T_{mm} \\[8pt]
+&= T_{cm} + (1+H) * T_{mm}
+\end{aligned}
+$$
+
+The additional $T_{cm}$ in the second term of the formula is called the **"cache search/lookup time"** because in the case of a cache miss the cache access is not for retrieving content from cache but to check for its existence.
+
+Cache search time is **zero** in the case for parallel access.
+
+In such a memory organization -
+- The Cache Memory is also called as the **Top Level memory**.
+- The Main Memory is also called as the **Bottom Level memory**.
+<h4 class="special">When to use which formula for Avg. Memory Access time?</h4>
+If a question has the words "cache memory access time" and "main memory access time" mentioned, only then move onto using the formulas for Simultaneous or Hierarchical access.
+- If "Hierarchy" or "Level" is mentioned, we are dealing with Hierarchical Access.
+- Else we are dealing with Simultaneous Access.
+
+Otherwise, if the question just mentions "time for cache hit" and "time for cache miss", use the [[Cache Organization#^a692b1|generic formula]].
+
+See [[Cache Organization#^q1|Question 1]] for a simple example.
+## Memory Access Time when Locality of Reference is used
+In the previous cases we were just looking at the cases where on a Cache miss we retrieve the data directly from the main memory. But on a cache miss, the block in which the data belongs to needs to be brought in the cache memory for future usage as well.
+
+Let the block transfer time be $T_{bt}$.
+
+Then,
+1. Simultaneous Access - $T_{avg} = H*T_{cm} + (1-H)*T_{bt}$
+2. Simultaneous Access - $T_{avg} = T_{cm} + (1-H)*T_{bt}$
+# Cache Write or Write Propagation
+Write propagation means that, if the CPU performs a write operation on some data in the cache memory, then that same data should also be updated in the main memory.
+
+![[Pasted image 20260125163924.png|550]]
+## Write Through
+If the CPU performs a write operation in the cache, it performs a write operation in the main memory **simultaneously/parallelly**. The data in the cache and the main memory is updated together.
+
+- **Pro -** No inconsistency between the content in the cache memory and content in the main memory.
+- **Con -** Time consuming because write operation is performed on the main memory irrespective of hit or miss in the cache.
+
+Because the cache and main memory are accessed simultaneously, the cache memory would use **simultaneous access**. Thus, the time required for one read and one write operation is -
+
+$$
+\begin{aligned}
+T_r &= H*T_{cm} + (1-H)*T_{mm} \\[8pt]
+T_{w} &= \operatorname{max}(T_{cm}, T_{mm}) = T_{mm} \\[8pt]
+T_{avg} &= \text{\% of read operations} * T_r + \text{\% of write operations} * T_w  \\[8pt]
+\text{Eff. Hit Ratio} &= \text{\% of hit read operations} = \text{\% of read operations} * H
+\end{aligned}
+$$
+
+Out of all cases of read/write operations hitting/missing, the cache memory is sufficient by itself only when a read-hit occurs. For rest of the cases the main memory is involved too. This causes a **lower effective hit ratio** for Write Through cache when compared with Write Back cache.
+## Write Back
+If the CPU performs a write operation in the cache, the same content in the main memory is not updated simultaneously. Instead the content in the main memory is updated when a block is replaced in the cache memory.
+
+- **Pro -** Time saving compared to Write-Through.
+- **Con -** Inconsistency between the content in cache memory and main memory.
+
+Any block that has been written over is called a **dirty/modified block**. For any block in the cache memory -
+1. If no write was performed on that block - Directly replace the block without any write in main memory.
+2. If write was performed on the block (If it's a dirty block) - Perform write back for the block.
+
+Unlike Write Through cache, there's no requirement of Write Back caches requiring strictly Simultaneous or strictly Hierarchical access. The average memory access time is -
+
+1. Simultaneous Access -
+
+$$
+T_{avg} = H*T_{cm} + (1-H)*(T_{bt} + \text{write back time})
+$$
+
+2. Hierarchical Access -
+
+$$
+\begin{aligned}
+T_{avg} &= H*T_{cm} + (1-H)*(T_{cm} + T_{bt} + \text{write back time}) \\[8pt]
+&= T_{cm} + (1-H) * (T_{bt} + \text{write back time})
+\end{aligned}
+$$
+write back time is - 
+
+$$
+\text{write back time} = \text{fraction of dirty blocks} * T_{bt}
+$$
+## Write Miss
+Write miss are handled in two ways, by using Write Allocate or by using No Write Allocate.
+### Write Allocate
+On a write miss the block is loaded into the cache and then written in the cache itself. Usually used with [[Cache Organization#Write Back|Write Back]] cache.
+
+Write Back cache with Write Allocate -
+1. Read -
+	- Hit - CPU reads content from cache.
+	- Miss - CPU reads content from main memory and brings the missing block to the cache memory by replacing an existing block **if needed**. If a dirty block is replaced, write-back to the main memory.
+2. Write -
+	- Hit - Perform write in cache.
+	- Miss - Bring the missing block to the cache memory by replacing an existing block **if needed** and then perform the write operation on it in the cache. If a dirty block is replaced, write-back to the main memory.
+### No Write Allocate
+On a write miss the block is written in the main memory and not loaded into the cache. Used with [[Cache Organization#Write Through|Write Through]] cache.
+
+Write Through cache with No Write Allocate -
+1. Read -
+	- Hit - CPU reads content from cache.
+	- Miss - CPU reads content from main memory and brings the missing block to the cache memory by replacing an existing block **if needed**.
+2. Write -
+	- Hit - Perform write in cache and main memory simultaneously.
+	- Miss - Perform write in main memory but do not bring missing block to the cache.
+
+---
+# Questions
+^q1
+<h6 class="question">Q1) If in a two level memory hierarchy, the top level memory access time is 8ns and the bottom level memory access time is 60ns, the hit-rate required is __ for the average access time to be 10ns. What is __?</h6>
+
+$\underline{\text{Sol}^n} -$
+Here as "memory hierarchy" and "two level" is mentioned, we are dealing with a hierarchical access cache organization. So,
+
+$$
+\begin{alignedat}{3}
+&&10 &= 8 + (1-H)*60 \\[8pt]
+&\Rightarrow&\,\,10&= 8 + 60 - 60H \\[8pt]
+&\Rightarrow&\,\,60H&= 58 \\[8pt]
+&\Rightarrow&H&= \boxed{0.967} \\[8pt]
+\end{alignedat}
+$$

content/Computer Organization and Architecture/Memory Organization.md

@@ -0,0 +1,22 @@
+Content V/S Data - 
+1. Data - What the programs ultimately wants to use.
+2. Content - Any value stored in the memory.
+# Associative Memory
+Associative memory retrieves data **by content instead of by address**. Hence it is also known as **content addressable memory**.
+
+ Unlike an addressable memory where the CPU supplies an address, in associative memory the CPU supplies a **search key**. Each cell in this memory holds -
+1. **Key/Tag -** What CPU knows
+2. **Value -** What CPU wants
+
+How it works -
+1. CPU sends a search key.
+2. The search key is matched with the keys of all cells **simultaneously in parallel**.
+3. The cell with the matching search key is identified.
+4. The **associated value** of this cell is sent back to the CPU.
+5. If multiple cells match with the search key, the associated value for both is sent.
+
+This memory is extremely fast, much faster than SRAM, but because the hardware requires the ability to compare keys of all cells in the memory simultaneously it is also very expensive.
+
+Uses -
+1. Caching
+2. TLB (Translation Lookaside Buffer) in OS

content/Discrete Maths/List of Mathematical Structures.md

@@ -0,0 +1,7 @@
+1. [[Group Theory#Algebraic Structure|Algebraic Structure]]
+2. [[Order Theory#Lattice|Lattice]]
+3. [[Group Theory#Semi-group|Semi-group]]
+4. [[Group Theory#Monoid|Monoid]]
+5. [[Group Theory#Group|Group]]
+6. [[Group Theory#Abelian Group|Abelian Group]]
+7. [[Vectors and Vector Spaces#Field|Field]]

content/Discrete Maths/Set Theory.md

@@ -1,5 +1,7 @@
 >[!SUMMARY] Table of Contents
 >- [[Set Theory#Set|Set]]
+>	- [[Set Theory#Mutually Exclusive Sets|Mutually Exclusive Sets]]
+>	- [[Set Theory#Collectively Exhaustive Sets|Collectively Exhaustive Sets]]
 >- [[Set Theory#Set Operations|Set Operations]]
 >- [[Set Theory#Principle of Inclusion and Exclusion|Principle of Inclusion and Exclusion]]
 >- [[Set Theory#Multiset|Multiset]]
@@ -10,7 +12,7 @@
 >		- [[Set Theory#Reflexive Relation |Reflexive Relation ]]
 >		- [[Set Theory#Irreflexive Relation|Irreflexive Relation]]
 >		- [[Set Theory#Symmetric Relation|Symmetric Relation]]
->		- [[Set Theory#Anti-Symmetric Relation|AntiSymmetric Relation]]
+>		- [[Set Theory#Anti-Symmetric Relation|Anti-Symmetric Relation]]
 >		- [[Set Theory#Asymmetric Relation|Asymmetric Relation]]
 >		- [[Set Theory#Transitive Relation|Transitive Relation]]
 >		- [[Set Theory#Equivalence Relation|Equivalence Relation]]
@@ -52,7 +54,18 @@ $$
 &= \boxed{2^n}
 \end{aligned}
 $$
+## Mutually Exclusive Sets
+A collection of sets $A_1, \dots, A_k$ is called mutually exclusive iff,
 
+$$
+A_i \cap A_j = \phi,\forall i\ne j
+$$
+## Collectively Exhaustive Sets
+A collection of sets $A_1, \dots, A_k$ is called collectively exhaustive iff,
+
+$$
+\bigcup_{i=1}^k A_i = U \qquad(U \text{ is the universal set})
+$$
 # Set Operations
 1. Union
 2. Intersection
@@ -214,6 +227,10 @@ $\qquad(OR)$
 
 A partition of a set $A$ is the grouping of all elements of $A$ into **non-empty subsets** such that every element only occurs in one subset.
 
+$\qquad(OR)$
+
+A partition is a collection of subsets of $A$ such that the subsets are both [[Set Theory#Mutually Exclusive Sets|mutually exclusive]] and [[Set Theory#Collectively Exhaustive Sets|collectively exhaustive]].
+
 1. Partition of $A=\phi=\{\}$ is $\phi=\{\}$.
 
 Check [[Set Theory#^q5|Question 5]] to see how to count the number of partitions of a set.

content/Mathematical Foundations of Generative AI.md

@@ -1,3 +1,16 @@
+>[!SUMMARY] Table of Contents
+>- [[Vectors and Vector Spaces#Vector|Vector]]
+>- [[Vectors and Vector Spaces#Field|Field]]
+>- [[Vectors and Vector Spaces#Vector Space|Vector Space]]
+>	- [[Vectors and Vector Spaces#Subspaces|Subspaces]]
+>- [[Vectors and Vector Spaces#Linear Combinations|Linear Combinations]]
+>	- [[Vectors and Vector Spaces#Affine Combination|Affine Combination]]
+>- [[Vectors and Vector Spaces#Linear Dependence|Linear Dependence]]
+>- [[Vectors and Vector Spaces#Span, Basis, and Dimension|Span, Basis, and Dimension]]
+>	- [[Vectors and Vector Spaces#Span|Span]]
+>	- [[Vectors and Vector Spaces#Basis|Basis]]
+>		- [[Vectors and Vector Spaces#Uniqueness of Representation Theorem|Uniqueness of Representation Theorem]]
+>	- [[Vectors and Vector Spaces#Dimension|Dimension]]
 # Vector
 What is a vector?
 - **The Physics definition -** A vector is a quantity with both magnitude and direction.

content/Mathematical Foundations of Machine Learning I/Linear Algebra/Vectors and Vector Spaces.md

@@ -0,0 +1,13 @@
+Probability is the study of certainty/uncertainty around any decision or action.
+# Random Experiment
+A random experiment is an experiment with a known set of outcomes, but the outcome of a trial is unknown before the trial is conducted.
+- This set of all possible outcomes of a random experiment is called the **sample space**.
+- **An event** is a subset of the sample space that is of our interest.
+
+**Parallels to Set Theory -**
+
+| [[Set Theory]] | Probability Theory |
+| :------------: | :----------------: |
+| Universal Set  |    Sample Space    |
+|     Subset     |       Event        |
+| Singleton Set  |      Outcomes      |