Digital Archives Lack Necessary Infrastructure Strategy
netzpolitik.org
- Much of the web is highly ephemeral, with the average webpage lasting only about 100 days.
- A lack of national strategy leaves digital history vulnerable, as 38% of websites from 2013 no longer exist.
- Reliance on a single source like the Internet Archive poses technical, legal, and political risks.
The Fragility of Digital Memory
- Digital historian Jens Crueger emphasizes that web archiving is critical infrastructure for historians, journalists, lawyers, and AI developers.
- The Internet Archive, while successful, faces pressure from mass data requests by AI firms and contains inherent risks due to its U.S.-based hosting.
- Archiving involves subjective choices; for instance, storing historical misinformation presents challenges when it is later used to train AI models.
The State of Archiving in Germany
- The German National Library (DNB) has a legal mandate but operates with limited scope, having conducted only one systematic crawl of .de domains in over a decade.
- Accessibility is severely hindered by the "terminal barrier," which requires researchers to visit specific library locations in Frankfurt or Leipzig to view the archives.
- Crueger advocates for removing this legal barrier to increase transparency and provide political incentive for better funding.
A Strategy for the Future
- There is an urgent need for a well-defined collection mandate that moves beyond ad-hoc event harvesting.
- A collaborative strategy should involve archivists, journalists, historians, and citizens to determine what content truly reflects society, similar to participatory approaches taken in Ireland.
- Digital preservation should be decentralized, encouraging individuals and groups to archive their own web presence, particularly for marginalized communities whose history is often the first to disappear.