Snacks
Posts
13158Following
561Followers
562Why do I want my liver to be eaten by a foxxy lady?
Typos are a spook
"Not the worst person on the instance"
@meso retrieval augmented generation. You use vectorization to categorize parts of your text and can then draw up the closest matches and feed just those to an llm
2
0
3
@meso you'll prob need some kind of rag setup of you want to query over your entire traffic law tbh. Maybe there's some rag in a box thing but i'm not aware of any
2
0
1
Snacks
repeated
repeated
@chjara oh, it's osu. Thought for some reason that's a fedi profile in the background
1
0
1
Snacks
snacks
Bicycle work is cool because i always do it better after the first time proving i'm not ocmpletely retarded and actually capable of learning
0
0
2
@meso usually llms are using 16bit weights nowadays but you can just use lower resolution
1
0
0
@meso just get a model that's smaller than the vram, usually you need about a gb less. Some 4bit quants of 12b and all of 8b models should fit
2
0
1
@meso sometimes, sometimes just a syllable or a single letter. Usually about 3-4 letters per token
1
0
1
@meso if you go to regular ram it's gonna get slow, like less than a token per second
1
1
1
@meso that's still enough for 4 bit quants of 12 or 8b models i think
1
0
1