p.enthalabs

An entire Herculaneum scroll has been read for the first time

scrollprize.org · Read Story HN original

Preprint: https://scrollprize.org/pdf/main.pdf

https://github.com/ScrollPrize/villa

Comments

I am on the vesuvius challenge team that did the segmentation, unwrapping, and ink detection, so feel free to ask any questions.
Given the current rate of progress, how long do you think it will take to decipher the entire collection?
That's a tough one to give a strong estimate of. Some scrolls are easier or harder to unwrap and read for a multitude of different reasons, mostly due to how damaged the scroll was in the eruption, and how easy or not the ink is to read. IIRC from what we've scanned of the herculaneum collection, none of the ink is easily visible via spectrum alone, so we have to use a lot of ML and physically based rendering techniques to be able to find ink. That also requires unwrapping and segmentation _before_ any ink detection.

For iron gall ink with high enough iron concentration, the ink stands out in the xray volume through simply masking off low values, such as was shown in our campfire scroll experiment a few years ago. No herculaneum scrolls show similar ink.

Thanks!
Do you think this particular scroll is easier or harder to read that the others will be? Or about average?
Pherc1667 was quite small and just so happened to have readable ink, so it was easier than I expect most others to be.
Do we known what ink is used?
Most of the evidence so far points towards carbon based ink. I am not sure if any of the scrolls we have scanned show strong evidence of iron gall based ink. I know that there are different types and preparation methods for different carbon based inks, but I do not know if it is possible to determine which kind(s) were used solely from inspecting the xrays.

I am, though, not a papyrologist, so historical ink making, preparation, and usage are not my field.

Thanks for answering all the questions in here. Fascinating work.
Are the fragments destroyed in ‘69 and ‘80 available to be read similarly? Or were they disposed of?
I am unaware of those fragments in particular. Though we have scanned a dozen or so fragments, mostly to help guide ink detection, since the ink in them is often more visible in visible and/or near IR light, but can be hard to impossible to detect in the xray spectrum.
What are the wildest, most exciting but plausible things that might be discovered in these documents?
I am not a papyrologist or a classicist, rather I'm a computer scientist, so my expertise is unfortunately not in _what_ the scrolls say, rather how we get there. That being said I think and hope that there will be a trove of things that has no known provenance at all, completely lost works that elude the public memory.
Well what were your first thoughts when you decoded the script, besides the obvious Eureka, after making some sense of the texts?
Probably something along the lines of "finally, now it looks like a coherent piece of text. I wonder what it says".
Other members that were on the team before me had already proved it out before I came along so I knew it was possible. The cool thing for me though was specifically doing some physicically based rendering techniques. How well these work varies greatly, but on a few segments in one scroll they work extremely well. I whipped up some simple code to composite layers, did up a render, and without any ML at all was looking at multiple rows of text that no one had read for 2000 years. That was neat.
Probably a lot more texts of Epicurean philosophy and not a whole lot else unfortunately according to my papyrologist friend.
Why would Epicurean philosophy be unfortunate?

I was under the impression that there was almost nothing left of that school of thought, and that it’s writings had been destroyed.

What would you like to have instead?

The unfortunate part is the lack of anything else therein, not that it's Epicurean philosophy.
The Jewish Talmud uses Epicurus's name as a term meaning "heretic".
The Epicureans were particularly hostile to the Jews and Christians, because Epicureans deny Providence or the active intervention of the divine in human affairs. See Horace Sermones 1.5.
It's more like the Christians and the Jews were particularly hostile to Epicureans and Stoics, because those mocked the claims about the existence of an all-powerful God that requires prayers.

The Epicureans and Stoics did not care much about Christians and Jews, but after the Christians obtained the power in the Roman Empire they made great efforts to persecute and discredit the Epicureans and the Stoics, as the most dangerous kinds of non-believers. (Unlike the rational Epicureans and Stoics, the traditional polytheists could be much easier converted to Christianity, by inventing a set of Christian saints to which the former polytheists could redirect the prayers and the holidays to which they were habituated.)

The Christian propaganda has created a false image of the Epicureans, which has persisted until today.

The Epicureans were not atheists, but they had a very different conception about what Gods are. They thought that in nature there are a lot of entities that have a god-like power, i.e. humans are too small and weak to influence them in any way, but the life of the humans is strongly dependent on the actions of those entities, so they can rightly be considered as gods. Examples of such entities are the Sun, the Moon, storms, volcanos etc.

Unlike in the traditional Greek and Roman religions, where it was believed that for each such natural phenomenon there exists some sentient god, who can be convinced to change the events to a more favorable outcome by prayers and sacrifices, the Epicureans believed that the gods, even supposing that they were sentient, in any case they do not care about humans more than humans care about ants, so there is absolutely no point in praying to them or bringing sacrifices to them.

Therefore humans should conduct their life according to ethic principles, but without worrying about what gods may think about their actions.

Many modern humans would probably agree with the Epicurean philosophy, which was completely different from what the Christian propaganda claimed, e.g. that Epicureans were some kind of sinners addicted to pleasures.

> completely different from what the Christian propaganda claimed, e.g. that Epicureans were some kind of sinners addicted to pleasures.

Interestingly, in Jewish literature (Talmud and further refined by Maimonedes) Epicurus refers to a certain kind of non-believer, not to a sinner for pleasure. See here for example https://www.sefaria.org/Mishneh_Torah%2C_Repentance.3.8?lang...

I always wondered about that because I guess I fell for the "Christian propaganda" as you call it.

Indeed, the 3 beliefs attributed to Epicureans there, i.e.:

a) one who denies the existence of prophecy and maintains that there is no knowledge communicated from God to the hearts of men;

b) one who disputes the prophecy of Moses, our teacher;

c) one who maintains that the Creator is not aware of the deeds of men.

are actually accurate enough renderings of what an Epicurean might have said in a discussion with a Jew, because as I have mentioned, Epicureans believed that there are gods, but those do not pay attention to humans and do not attempt to communicate with humans, because humans are insignificant for them.

This is quite different from how Epicureans were portrayed in Christian literature, where calumnies against them were preferred for avoiding any direct controversy.

> What would you like to have instead?

History! That's what intrigues me the most: texts with accounts of events that have otherwise vanished from the historical record.

Past events, and the ideas behind them, are both first-class history topics.
That's what was thought, but maybe not -- only one of the three so far looks Epicurean, which is not what was expected. Maybe it's a fluke, but historians are buzzing a bit about whether it might be broader than expected.
in the paper it says "The recovered text is a philosophical treatise on ethics, and the evidence points to a Stoic work: it turns on human nature, impulse, and the moral progress of human beings, and its final preserved column names Aristocreon — nephew and disciple of the great Stoic Chrysippus — which, together with the language and themes of the text, places it in a Stoic context and dates it to the 2nd century BC."
Woah there was a lost Homer epic comedy about a bumbling fool named Margites?
There's also the Telegony. Odysseus has a son through Circe who winds up killing him and marrying Penelope. Odysseus son through Penelope, Telemachus, marries Circe. There's some wild stuff that doesn't survive.
Looking through these it’s crazy to find out that The Iliad is only 1 of like 5 original texts on the Trojan war. We’re reading book 2 of a 5 book series
It was an oral epic passed through generations for quite a while before anything was written down so there isn't necessarily much of an "original"
Aristotle's second book of Poetics, of course.
we already know that a blind Italian monk burnt it to ashes, at least, that's what Eco wrote and he was a learned scholar
but that was a copy
well the other existing copy (or original) was destroyed with the library of Alexandria
Did anyone on the team come from a non-science, non-math, non-academia background? Did anyone working on this just teach themselves and start contributing?
Yes. Sean, who was a co-winner of the 2024 prize, IIRC has no formal background in ML, computer science, AI, etc. He is one of our core researchers and the most productive team member.
I've been on the Discord for a couple of years now, and poking around with submissions as well. Sean and the entire team deserve so much praise for all of this work.

It's easy to just read about the breakthrough and see it as one neat, linear line to get there, and hard to comprehend the hours, months and years that so many spent to get there. Big congrats to you, Sean, Nat and the entire team!

That's incredibly impressive.

Major kudos to all of you on your achievements! This is amazing work for anthropology and for society, and it's greatly appreciated.

How do get to do that? As in what did you study to get the prerequisite knowledge, and how did you find this particular job? When I see interesting jobs I'm anyways curious what path lead there
I am a computer scientist. I studied CS in university, worked in the semiconductor industry for a while, got started as a participant in the challenge aspect of the Vesuivus Challenge. They were hiring, I sent in an application, interviewed, and was offered the job.
That last sentence is so perfect, like my dad answering the question of how he lost weight. "I ate less and exercised more."
Very cool that you got in just through your interests. It's anyways cool to see stories where that works out! Good to know it's possible to get that kind of job
I don't have any questions, just a comment.

You have a potential to rewrite the history of European Antiquity quite substantially. The Herculaneum set of scrolls is enormous and must contain a lot of hitherto unknown.

That comes with a set of peculiar risks. Once your work starts producing something that contradicts previous work of Very Important People, they will lobby to stop you. Be prepared for that.

Science should be neutral and always value new evidence. Scientists as humans are unfortunately subject to all sorts of passions.

What contradictions do you think the scrolls contain?
I don't have any concrete tips.

We have very little written material surviving from Rome, at least from the period before a codex (book) was invented, which was more durable that a scroll. Often, we only know of one source describing important events, and when it comes to political struggles and civil wars, the perspective of the defeated party often did not survive. The punishment of damnatio memoriae was practised and even among the early emperors, Caligula and Nero were subject to a form thereof. (This library in Herculaneum was buried 11 years after Nero's death.) I would be surprised if everything in the scrolls perfectly aligned with the record that survived for 2000 years and that was filtered by both random chance and political/religious censorship. Even Christians later destroyed some pagan texts.

BTW personally, I would love for some textbook of Etruscan to emerge from there. This was once again a language whose teaching was banned in Rome.

No questions, but I just want to say this is really exciting work!
Outstanding work! I've participated in the challenge, but didn't get far. One of the questions I had at the time was - if I'm going to use ML to detect ink, could it invent hallucinated letters, or even parts of text, and how to prevent that?
Yes, it's quite possible for ML to hallucinate ink, though it is on a much more local scale, like predicting a slightly longer stroke, filling in more of a character than is actually in the data, etc. Perhaps enough to change a reading of a character or show where ink isnt. It is difficult for ink detection to hallucinate grammatical and idiomatic greek and latin.
What is the input to the ML algorithm? Does it know the surrounding context so that it has a chance to deduce "if this stroke is slightly longer then the end result will be idiomatic greek and latin"?
Not all machine learning is generative AI.
True but like regular document scanning software there can be errors in detection.
Just as with redacted documents (consistently blocked terms) or bad OCR jobs (wrong or missing characters), even if only a certain percentage comes out unmangled it is more readable than having no data at all.

A stable base corpus and some dynamic programming will allow you to clean up the remainder[0].

[0]: http://stackoverflow.com/a/11642687/2449774

Amazing work, fantastic!
Absolutely incredible work. This is one of the most amazing news articles I’ve encountered in decades. Congratulations team!
this is überragend. by many means!
I'm interested to know about the approaches that you tried with the ML, and then decided to not use. In practice, the options are so many. How did you come up with the final approach - and was there a systematic way to decide which options to go for?
I am not on the research team, rather on the production side of things, so my knowledge on that is pretty limited. I think one of the main takeaways from a lot of the research, though, on both the segmentation side and the ink detection side, is that it's a lot less about what models and techniques and such you use, but how good your training data is. Gathering ground truth is hard, and if you don't have a lot of good ground truth, it doesn't matter if your code is perfect, you'll never get results.
> it's a lot less about what models and techniques and such you use, but how good your training data is.

Ah, the good old bitter lesson strikes again

That is a general truth of most ML; many models _can_ find the information in the data, if the data is good enough. If it is not, then likely no model can.
You brought up what I'm most curious about: Where does the ground truth come from for this work since you can't just to unwrap a scroll to tell if the model got it right or, presumably, make a facsimile scroll and wrap it up.
The ground truth comes from manual work. The scrolls can be unwrapped virtually, manually, through extensive pointing and clicking by a human on the boundaries of the scroll. This, in and of itself, is not particularly hard in sections of the scroll that are preserved well, but is extremely tedious and slow and error prone. We have a team of annotators who do manual annotation and refinement through custom software we've written, mostly improving on automatically generated segmentations and unwrappings.

Once you have some unwrapped papyrus, you can render it to an image and look for ink. Ink leaves a certain texture that can be identified by the naked eye and labeled. Between these two processes you get the segmentation and ink detection ground truth. Segments can be flattened virtually through existing software and algorithms.

I'm sure that process is described somewhere on the project's site and, being a lazy human (and unwilling to ask LLMs to summarize it for me), I leaned on you for a human answer. I really appreciate you taking the time to answer. Thank you.

I can see why you'd be attracted to this project from a "let's solve problems computationally" perspective (never mind the historical side). It sounds like there are some cool problems in there.

The eye toward automating the process that the project seems to be targeting is particularly cool, too. This kind of stuff that makes me have real enthusiasm for ML.

how many scrolls have been scanned so far? what's the main limitation on scan amount?

have any attempts (or just ideas) been made to recreate such charring on known texts?

30 scrolls, maybe? Something like that. I scanned Pherc Paris 4 and Pherc Paris 3 at Beam line 18 at ESRF back in March.

The team did "the campfire scroll" experiment a few years ago to replicate carbonization, unrolling, and ink detection. That is the only case I am aware of. It proved the method could work but it's not a source of say training data; it varies too much from the real scrolls.

The main limitation is time and cost. We have to scan on what is AFAIK the most powerful x-ray beam line in the world. It is not cheap

You had to pay? I understand the machine cost many hundreds of millions of dollars, but I would have thought for academic researchers doing open science, the beamtime is free (funded by the govt / science trusts).
The beam time is unfortunately not free. I scanned Pherc Paris 4 and Pherc Paris 3 in March and had the final shift on the beam. As I was removing the scroll from the scanning pedestal the next team of scientists were already in the lab getting their samples ready. It's a well oiled machine and they've got customers.
What other type of stuff gets scanned? I can’t imagine a whole industry waiting to x-ray something?
Materials science, battery research, pharma (protein structure)
The way these things normally work is that the project starts with some sort of a grant. Then that grant pays for all of the costs of the project: peoples' salary, materials used, time on equipment, plus money for the buildings and administration (overhead).

In this case the time on the equipment would need to be included, both a portion of the cost of building/maintaining it, and probably the energy needed to run it. Even where the government is providing the grant (likely here), it still needs to be accounted for.

We - the core challenge team anyway - get no money from any government. We paid for the beam time from our donations and internal funding.
How fast is the process?

Could it be automated to the point where it's faster to scan a book closed than opened?

We've been trying to automate since the beginning. A lot of it is automated but it's mostly the easier and less damaged parts of the scrolls. Scanning takes a few days for the biggest scrolls but the amount of human refinement is still a multi month process.
may you please tell us how much effort goes into each type of task in those months?

where else do you think these techniques be applied?

We are a core team of about 10 researchers and developers working full time on work that applies to all of the scrolls. We also ahve 4 full time annotators that tend to work on one scroll at a time. The amount of time spent on any given scroll varies with how difficult and large it is.

There is an extremely large overlap between a lot of the work we do with medical imaging, CT scanning, XRay technology, and such. A lot of the ML models and frameworks we have used and adapted for our purposes originated in the medical field for things like cancer detection or segmenting different body parts.

Random shower thought: I wonder if it would be better in the long term to stop digging out archeological findings. The more we excavate, the more damage we do for future archaeologists who will have the superpower of reading these texts without even needing to dig the scrolls free and open them.
Modern archaeologists are painfully aware that theirs is a destructive science, and do their best to mitigate that. The most extreme example is probably the tomb of the First Emperor, Qin Shi Huang, where official policy on excavation can be boiled down to "not yet".
We stand on the shoulders of those that came before us. People have been trying to unroll and read the scrolls for 250 some odd years now. Had they not laid the groundwork for all that time we wouldn't be making the progress we are now.
Archaeologists think about this a lot. Many digs leave portions intact specifically so that future scientists, with access to techniques and technologies beyond what's available now, can research them.
How many scrolls are intact (worldwide, rather than just France) that might still be recoverable?
IIRC 99% of all of the existing scrolls are still in Italy's possession. I think the breakdown is something like ~350 are mostly in tact, another ~1000 are damaged but still "scroll like", and the remaining hundreds are shattered fragments.
...plus the ones that have not been dug out yet... the site is still partially buried
My god, but that sounds wonderful.