Krüppel-associated box (KRAB) zinc finger proteins are a large class of tetrapod transcription factors that usually exert transcriptional repression through recruitment of TRIM28/KAP1. The evolutionary root of modern KRAB domains (mKRAB) can be traced back to an ancestral motif (aKRAB) that occurs even in invertebrates. Here, we first stratified three subgroups of aKRAB sequences from the animal kingdom (PRDM9, SSX and coelacanth KZNF families) and defined ancestral subdomains for KRAB-A and KRAB-B. Using human ZNF10 mKRAB-AB as blueprints for function, we then identified the necessary amino acid changes that transform the inactive aKRAB-A of human PRDM9 into an mKRAB domain capable of mediating silencing and complexing TRIM28/KAP1 in human cells when employed as a hybrid with ZNF10-B. Full gain of function required replacement of residues KR by the conserved motif MLE (positionsA32-A34), which inserted an additional residue, and exchange of A9/S for F, A20/M for L, and A27/R for V. AlphaFold2 modelling documented an evolutionary conserved L-shaped body of two α-helices in all KRAB domains. It is transformed into a characteristic spatial arrangement typical for mKRAB-AB upon the amino acid replacements and in conjunction with a third helix supplied by mKRAB-B. Side-chains pointing outward from the core KRAB 3D structure may reveal a protein-protein interaction code enabling graded binding of TRIM28 to different KRAB domains. Our data provide basic insights into structure-function relationships and emulate transitions of KRAB during evolution.