Hacker News
new
|
past
|
comments
|
ask
|
show
|
jobs
|
submit
login
xcodevn
10 months ago
|
parent
|
context
|
favorite
| on:
Show HN: We've open-sourced our LLM attention visu...
On a related note: recently, I released a visualization of all MLP neurons inside the llama3 8B model. Here is an example "derivative" neuron which is triggered when talking about the derivative concept.
https://neuralblog.github.io/llama3-neurons/neuron_viewer.ht...
skulk
10 months ago
|
next
[–]
This is insanely fun to just flip through. I found a "sex" neuron.
https://neuralblog.github.io/llama3-neurons/neuron_viewer.ht...
vpj
10 months ago
|
prev
[–]
Pretty cool. The tokens are highlighted based on the activation?
xcodevn
10 months ago
|
parent
[–]
Yes, you're correct. The tokens are highlighted based on the neuron activation value, which is scaled to a range of 0 to 10.
Join us for
AI Startup School
this June 16-17 in San Francisco!
Guidelines
|
FAQ
|
Lists
|
API
|
Security
|
Legal
|
Apply to YC
|
Contact
Search:
https://neuralblog.github.io/llama3-neurons/neuron_viewer.ht...