This is the 26 categorical finetune of nomic-bert-2048's encoder.

130,000,000 - 4-30 masked token samples with 80% mask rate

253,952,000 - 77 token samples with 20% mask rate

775,000,000 - 144-256 token samples with 30% mask rate

453,800,000 - 385-512 token samples with 30% mask rate

Total samples; 1,228,800,000

The model has learned to categorize certain masked patterns with their categories and special tokens.

She's still cooking, but her accuracy is getting really up there. I plan to run the 256-512 token counts for a considerable amount of time and then we'll upgrade to 512-1024, and finally 1024-2048.

<subject>
<subject1>
<subject2>
<pose>
<emotion>
<surface>
<lighting>
<material>
<accessory>
<footwear>
<upper_body_clothing>
<hair_style>
<hair_length>
<headwear>
<texture>
<pattern>
<grid>
<zone>
<offset>
<object_left>
<object_right>
<relation>
<intent>
<style>
<fabric>
<jewelry>

With the categorical shunts;

[SHUNT_1000000]
[SHUNT_1000001]
[SHUNT_1000002]
[SHUNT_1000003]
[SHUNT_1000004]
[SHUNT_1000005]
[SHUNT_1000006]
[SHUNT_1000007]
[SHUNT_1000008]
[SHUNT_1000009]
[SHUNT_1000010]
[SHUNT_1000011]
[SHUNT_1000012]
[SHUNT_1000013]
[SHUNT_1000014]
[SHUNT_1000015]
[SHUNT_1000016]
[SHUNT_1000017]
[SHUNT_1000018]
[SHUNT_1000019]
[SHUNT_1000020]
[SHUNT_1000021]
[SHUNT_1000022]
[SHUNT_1000023]
[SHUNT_1000024]
[SHUNT_1000025]

Each shunt meant to activate cross-categorical conceptualization within their 77 token window.

Downloads last month
119
Safetensors
Model size
137M params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for AbstractPhil/bert-beatrix-2048

Finetuned
(2)
this model

Space using AbstractPhil/bert-beatrix-2048 1