Sentiment Model

A real, small character-level transformer that reads one headline's text and predicts a percent-scale forward stock-price return — the model behind the tape transformer's own "lm sentiment" feature (see /tape-transformer). Distinct from the Language Playground's text generator (see /language-playground): this model only ever reads text and outputs a single number, it never generates text. Unlike the other two research models, this one previously trained ONLY via cron's daily gate with no manual controls — this page gives it the same "Train more" and "Try it" testing tools.

Back to tape transformer

Model status

Trained steps

45,201

Train loss

0.000611

real mean squared error on the forward-return target, this chunk

Validation loss

0.002273

never trained on — the real generalization check

Checkpoint

Restored from DB

Best validation loss

0.000633 (step 27)

Try it below always uses THIS snapshot, never the live training weights above

Training examples

109

a (headline, forward-return) pair only counts if a real tape_history price exists at both publish time and 2 days later

Validation examples

Real headlines in window

921

stock_news, last 180 days — the raw pool before the price-match requirement above narrows it down to real training examples

A real transformer encoder over character-level headline text: d_model=16, heads=4, layers=2, feedforward=64, max length=96 characters. Trained with plain SGD (learning rate 0.01, weight decay 0.0001) to minimize squared error against the real forward 2-day return of the symbol each headline is about. Deliberately NOT wired into live Top Pick scoring directly — only a persisted history of its own real reads (lm_read_history) is, so a live click here never affects a real trading decision.

Training loss curve

trainvalidation

step 44193 — loss 0.00105step 45201 — loss 0.00061