Tag: #embeddings36 results
A discussion on the mathematical and intuitive reasons why Word2Vec output layer weights function as meaningful word embeddings. It addresses common confusion regarding why prediction parameters evolve into semantic representations.
512-dim 32-bit ArcFace embeddings exceed Postgres TOAST threshold, causing extra I/O via TOAST tables. Suggests 16-bit HALFVEC quantization halves storage/I/O while fitting inline. Expects negligible real-world precision loss.
Practitioner used ArcFace/CosFace metric learning on 30 training classes to handle many more at inference via embeddings and cosine similarity clustering. Approach succeeds on unseen classes without rejecting as OOD. Post seeks standard ML term for this paradigm.
Matryoshka Representation Learning excels in maintaining performance under embedding compression. However, recent work shows degradation in retrieval tasks. Community seeks papers and experiments on other limitations.
Developer created a 3D visualization of ~7,000 High Court cases using embeddings, citations, and clustering from Open Australian Legal Corpus processed by Kanon 2 tools. Nodes sized by citation frequency, colored by citation sentiment, clustered by topics like estate and criminal law. Includes tutorial on replicating with own documents.
Post seeks methods to fine-tune sentence embedding aggregation beyond mean pooling from transformers, without labels or changing the base model. Focus on general self-supervised strategies for non-NLP datasets. Bonus interest in dimensionality reduction.