NEWS

Rhobots 0.1.10 (2026-08-08)

embed_texts(chunk_strategy = "mean" / "first") treated padding as real text: short documents were padded to the longest document in the batch, the padding was attended to, and a pad token was used as [SEP]. Chunked embeddings change as a result and now match embedding each document on its own. Re-run any chunked embeddings made with earlier versions.
hdbscan_clustering() with the default allow_single_cluster = FALSE returned all documents as one cluster when the cluster tree never split into groups of at least min_pts. It now returns all documents as noise, as Python hdbscan and dbscan::hdbscan() do.

embed_texts() uses far less memory on large corpora. On 3,000 documents (MiniLM, batch 64 x 256 tokens) peak memory fell from 17.6 GB to 4.4 GB, and to 1.9-2.4 GB with max_tokens, without slowing down. Peak memory now depends on the largest batch, not on corpus size. Default output is unchanged.
- Finished batches and each encoder layer's attention tensors are freed straight away; torch tensors are invisible to R's garbage collector, so they used to pile up. New argument gc_every (default 1; 0 turns all cleanup off).
- New argument max_tokens caps padded tokens per batch, so long texts get smaller batches and short texts can use a large batch_size.
- Batches run longest first, so the first batch reserves the largest memory blocks and later batches reuse them. An out-of-memory error therefore appears at the start of a run, not the end.
- Texts are tokenized and chunked in blocks of documents, and the result matrix is no longer copied during reordering or normalisation.

DESCRIPTION rewritten to spell out all acronyms in full (Uniform Manifold Approximation and Projection, Hierarchical Density-Based Spatial Clustering of Applications with Noise, term frequency-inverse document frequency, part-of-speech) and remove author-name citations, eliminating all spell-checker NOTEs in the automated CRAN incoming feasibility check.
nanoflann.hpp moved from inst/include/ to src/ (internal use only); removes the 'cc not on path' NOTE on Windows by taking it out of the public header path that R's compiled-code checker inspects.

All \dontrun{} example blocks replaced with \donttest{} (examples require a model download and cannot run in < 5 s; \dontrun{} retained only for rhobots_install() which installs the torch backend).
Software and package names in DESCRIPTION now consistently wrapped in single quotes per CRAN policy.

load_specter2() now correctly injects adapter weights into all transformer layers. The previous code used a nested replacement chain (enc$model$encoder$layer[[i]]$adapter <- adp) which R desugars into a [[<- write-back that can reinitialise the submodule; weights from earlier layers were silently overwritten by each successive iteration. Fixed by pre-fetching direct layer references (lapply(seq_len(n_layers), \(i) ...)). SPECTER2 embeddings now match the Python reference (cosine similarity > 0.9999).
load_hf_bert() / load_specter2() now correctly read safetensors files named pytorch_model.safetensors in addition to model.safetensors.
weight_loading.R now detects ZIP-archive .bin files (PyTorch 1.6+ format, magic bytes PK) and emits a clear error message with conversion instructions, in addition to the pre-existing guard for legacy pickle format (0x80).
torch::torch_load() call updated from deprecated map_location argument to device (renamed in torch 0.16.3).
Default repositories for load_specter2() updated to NetworkIsLife/specter2_base and NetworkIsLife/specter2, which host safetensors versions of the original allenai/specter2_base and allenai/specter2 (the AllenAI repos only provide .bin files that R torch cannot read directly).

hdbscan_clustering() gains allow_single_cluster (default FALSE, matching Python hdbscan's default). When FALSE, EOM never returns a single root cluster when sub-structure is present in the condensed tree. This fixes cases where diffuse embedding spaces (continuous sentiment data, overlapping topics) collapsed to 1 cluster. On the validation suite, datasets that previously returned 1 cluster (Emotion, Yelp Reviews, IMDB, Rotten Tomatoes) now match Python's cluster counts within ±1.
topic_quality() gains a space argument ("original" / "reduced"). When "reduced", cohesion, separation, and silhouette are measured in the UMAP-reduced space rather than the full encoder embedding. This makes quality metrics much more sensitive to different UMAP/HDBSCAN parameter choices and is the recommended setting when using sweep_topics().
sweep_topics() gains a corresponding quality_space argument that is forwarded to topic_quality() (default "reduced").