MyHeritage operates at the intersection of Big Data, consumer genomics, computer vision, and historical record linkage. Processing tens of millions of DNA samples alongside billions of historical records requires a robust infrastructure capable of handling massive petabyte-scale datasets, complex graph processing, and computationally intensive machine learning workloads.
1. High-Throughput DNA Microarray and Next-Generation Sequencing Infrastructure
The DNA pipeline handles raw data generated primarily through Illumina Infinium Global Screening Array (GSA) chips. These microarrays process hundreds of thousands of single nucleotide polymorphisms (SNPs) per sample.
[Raw Microarray / FASTQ Data]
│
▼
[Quality Control & Filtering]
│
▼
[Statistical Phasing (BEAGLE / SHAPEIT)]
│
▼
[Genotype Imputation Platform]
│
▼
┌─────────────┴─────────────┐
▼ ▼
[Admixture Ethnicity Pipe] [IBD Matching Engine]
Raw Data Processing & QC
Incoming genotype files undergo strict quality control (QC) filtering, excluding SNPs with high missingness rates ($>5\%$), low Minor Allele Frequency ($\text{MAF} < 0.01$), or deviations from Hardy-Weinberg Equilibrium ($p < 10^{-6}$).
Phasing and Imputation
Unphased genotype data is resolved into maternal and paternal haplotypes using statistical phasing tools like BEAGLE or SHAPEIT. Following phasing, MyHeritage utilizes an imputation engine to infer unobserved genotypes against a reference panel (such as the 1000 Genomes Project or internal high-coverage panels), expanding the raw SNP count from ~700,000 markers to over 10 million markers for enhanced resolution in downstream matching.
2. Distributed Algorithms for Identity-by-Descent (IBD) Detection
Finding biological relatives across a database of tens of millions of users requires pairwise comparisons that scale quadratically ($O(N^2)$). MyHeritage addresses this bottleneck using scalable parallelized workflows.
[Target Haplotype Array]
│
▼
┌───────────────────────────────────┐
│ Positional Burrows-Wheeler Trans. │
│ (PBWT) Matches Shared Segments │
└───────────────────────────────────┘
│
▼
┌───────────────────────────────────┐
│ Germline Segment Stitching │
│ (Allowing Minor Match Gaps) │
└───────────────────────────────────┘
│
▼
┌───────────────────────────────────┐
│ Genetic Distance Map Conversion │
│ (Centimorgans - cM) │
└───────────────────────────────────┘
Positional Burrows-Wheeler Transform (PBWT)
To make $O(N^2)$ comparisons computational feasible, MyHeritage leverages PBWT algorithms. PBWT sorts haplotypes at each genomic position based on their prefix matches, allowing shared identical-by-descent (IBD) segments to be discovered in linear time relative to database size ($O(N)$ per query marker).
Genetic Distance Calculation
Shared segments identified via PBWT are measured in centiMorgans (cM) using recombination maps (such as the Haldane or Kosambi map functions):
$$d = 100 \int_{x_1}^{x_2} r(x) \, dx$$
Where $r(x)$ represents the recombination rate per base pair. Segments below a strict length threshold (typically $<6\text{ cM}$) are filtered out to reduce false positives resulting from ancient population bottlenecks (Identical-by-State / IBS).
3. Deep Learning in Historical Image Restoration and Enhancement
MyHeritage integrates computer vision models into its consumer family history stack to enhance, colorize, and animate historical photographs.
| Feature Tool | Core AI/ML Architecture | Technical Function |
| MyHeritage InColor™ | Deep Convolutional Neural Networks (CNNs) / GANs | Predicts realistic chromatic channels ($a^*, b^*$) in $L^*a^*b^*$ color space from grayscale input ($L^*$). |
| MyHeritage Photo Enhancer | Generative Adversarial Networks (GANs) / Super-Resolution | Increases spatial resolution and restores low-frequency facial details via deep spatial priors. |
| DeepStory / Deep Nostalgia™ | Motion-Driven Video Generation (e.g., First Order Motion Models) | Maps facial keypoints from source driver videos onto target static photos to simulate life-like motion. |
4. Graph-Based Historical Record Linkage Engine
The platform manages an extensive database containing billions of historical records—such as censuses, birth certificates, and passenger lists—alongside millions of user-contributed family trees.
[User Tree Nodes] [Historical Record Entities]
│ │
└───────────────┬────────────────┘
▼
┌───────────────────────┐
│ Record Matching Engine│
└───────────────────────┘
│
┌────────────────┴────────────────┐
▼ ▼
[Deterministic Parsing] [Probabilistic Models]
• Soundex / NYSIIS • Fellegi-Sunter Methodology
• Exact Name Indexing • Graph Traversal Similarity
Entity Resolution Mechanics
- Phonetic and String Metrics: Names are processed using algorithms tailored for historical noise and variation, such as Soundex, NYSIIS, or Jaro-Winkler Distance metrics.
- Probabilistic Matching Framework: Based on the Fellegi-Sunter methodology, matches evaluate matching fields (e.g., date of birth, geographic location, relatives’ names) against non-matching configurations:
$$w = \log_2 \left( \frac{m_i}{u_i} \right)$$
Where $m_i$ is the probability that field $i$ agrees given the entities are identical, and $u_i$ is the probability that field $i$ agrees purely by chance.
- Graph Traversal Matching: Family trees are modeled as directed acyclic graphs (DAGs). The system evaluates neighborhood subgraphs around an individual, calculating context-aware confidence scores for potential historical matches.
5. Security Architecture, Privacy Controls, and Compliance
Handling sensitive biometric and genealogical data necessitates stringent compliance (GDPR, CCPA) and specialized security practices.
┌─────────────────────────────────────────────────────────────┐
│ Client API Request │
└──────────────────────────────┬──────────────────────────────┘
│ (TLS 1.3 / mTLS)
▼
┌─────────────────────────────────────────────────────────────┐
│ Security Perimeter Gateways │
└──────────────────────────────┬──────────────────────────────┘
│
┌──────────────────┴──────────────────┐
▼ ▼
┌───────────────────────┐ ┌───────────────────────┐
│ User Metadata │ │ De-Identified │
│ (Relational DBs) │ │ Genomic Big Data │
└───────────┬───────────┘ └───────────┬───────────┘
│ │
▼ ▼
┌───────────────────────┐ ┌───────────────────────┐
│ AES-256 Envelope Enc. │ │ Salted Hash Alias │
│ (Data-at-Rest Protection) │ (Zero Direct PII) │
└───────────────────────┘ └───────────────────────┘
Data Anonymization Architecture
Genomic processing pipelines strictly isolate personally identifiable information (PII) from biological sequence data. Raw DNA files and processed genotype matrices are indexed via pseudonymous cryptographically generated IDs.
Cryptographic Standards
Data in transit across microservices utilizes TLS 1.3 protocols with mutual authentication (mTLS). Storage infrastructure enforces AES-256 envelope encryption at rest, backed by Hardware Security Modules (HSMs) for cryptographic key rotation.
6. High-Scalability Database Architecture
To deliver real-time insights to millions of concurrent users, MyHeritage utilizes a hybrid data layer architecture designed to minimize latency across diverse workloads:
- Relational Layer: Sharded relational databases manage user accounts, subscriptions, and relational family tree models.
- NoSQL Key-Value / Columnar Stores: High-throughput storage systems (such as Apache Cassandra or HBase) host genetic matching matrices and IBD segment lookups to allow fast parallel reads during user sessions.
- Search Clusters: Distributed search clusters powered by Elasticsearch provide sub-second full-text index querying across billions of transcribed historical records.
Also Read: The Evolution of the Modern Trail: A Deep Dive into Spypoint Technology – My Tech Blaze
Source: Free Family Tree, Genealogy, Family History, and DNA Testing
