Introduction
Today's post I would like to introduce a project I have been working on called Hive Legacy.
We have seen projects, various frontends and communities come and go. Those links that worked years ago , now lead to nowhere even though the post or comment data is still stored on the blockchain.
That is where the idea came to me. We need a way to preserve access to the content stored on chain. To preserve the information that helps us find that content.
The simple question that started it all.
Yep, it was one question that got everything rolling.
What happens to a person's Hive history if the frontend or website used to access it disappears?
We hear quite a bit how user's data is stored on the Hive blockchain and that is a good thing. Yet , I personally have seen multiple frontends come and go for various reasons like:
- Frontends have shut down
- Domains expire or change
- Apps or project stop being maintained
- Communities move or are no longer active Frontends that were popular in the early days of the blockchain like eSteem, Busy.org, Dlive, VIMM, Steepshot and many others are now gone. Also many communities had frontends just for themselves that have the same issues, no longer maintained or an expired domain. All of those links out in the world are broken.
The content of those posts and comments still exist on chain, but the path people used to get there is now gone.
What we have is essentially Link Rot.
Why this bugged me.
The idea of link rot really did bug me. It's one thing that very many links of the Web 1.0 and 2.0 era are long gone. Not just the fact that the links no longer work, but that the data is gone too.
But this is where Hive is supposed to be different, where all the data is safely stored on chain. But what use is it , if all the links to a user's post are no longer valid? How would someone in the future find a Hive post that was of topical significance to them if the link they found pointed to a frontend that died years ago?
Or imagine someone in the future researching a particular historical event, like the Steem Takeover, when there are no longer any frontends? What if there are no longer any witnesses producing blocks and all that is left are a few copies of the blockchain sitting in some digital archive? Traditional hyperlinks are not totally useless, but it would be a tough road to follow.
The idea of an "archive".
The idea that link rot could hide a good portion of the comment data stored on Hive, making it essentially undiscoverable, just didn't sit well with me.
I wanted a way for a Hive user, or any user, to "archive" a users post and comment history. What that account published to the blockchain. A persons writing history on Hive not just preserved, but still accessible.
Preserving the ACCESS to the data and not the actual data.
Hive is the blockchain that preserves all of the underlying data and records. In no way is Hive Legacy meant or intends to be a blockchain replacement or to make a copy of the blockchain.
Hive does a great job at storing a user's data, but the goal of Hive Legacy is to preserve the information needed to locate and understand that data.
The Hive "Card Catalog".
I would like you to think of Hive Legacy as a library card catalog. The library stores all the books in one large building and all the books are organized by various pieces of information, like title, subject, author and so on. Imagine you wanted to find everything written by a particular author about fishing. You would go to the library because that is where the author's books on fishing are located. People know this, libraries are where the books are stored and organized.
But most people can't just walk into a library and know for sure where the author's books on fishing are located. They know they are there, just not exactly where they are located.
That is what the card catalog is for. It is to help a reader to locate a particular book or books by subject , title, author, date and so on. The reference card is NOT the book, it is the pointer to where the book is located in the library. And not just that, it also provides basic information about what a book is about without someone having to read or skim through the actual book.
The same can be thought of about Hive.
Creating an archive of a users history.
This is where Hive Legacy was born.
I started thinking : How could I preserve a user's history on Hive in a portable record?
At first I thought about either a plain text or markdown file but finally decided on JSON. Most modern programming languages have ways to handle JSON elegantly. And JSON is an established format that is widely accepted. So now I have the storage format, now the big decision, what goes in it?
What information goes into a Hive Legacy Archive?
Since my original concern was link rot of a Hive frontend that is no longer active or working, I decided early on I was not going to try and store full post and comment data of a user. This is the job of the Hive blockchain to store that kind of data. What I wanted preserved was the reference to the user's stored data, a pointer, the breadcrumbs trail, the card catalog in the libraries. I didn't want to copy all the books of a particular author, but instead record what they are and where they are located.
The archive needed the information stored in it to serve the function of that crucial pointer to a user's data . I came up with a list of items that would be vital in helping someone in the future, who came across such an archive, find the information being referenced. What I came up with was:
- Account information: The all important account name.
- Post/Comment title: The title of a user's post
- Date/Time: When the user's post/comment was broadcast to the blockchain.
- Permlink: This is the reference to a user's post/comment on the blockchain.
- Tags and Categories: Information about what a post's subject matter and if it was posted in a community or not.
- Application/Frontend: What app or frontend the user used to make a post or comment with.
- Transaction ID/ Block number: Additional reference points for locating a user's posts/comments. (Added later through the enrichment process)
I felt with these pieces of information, someone could find a user's history on Hive, even if the existing frontends are no longer in operation. Since the archive is in the JSON format, a tool could be developed or with the assistance of AI, someone could retrieve that information from a user's archive.
What can you do with a Hive Legacy archive?
Originally, I wanted to archive a user's posting and comment history without relying on a frontend still being operational. The idea was that the archive could be used to reconnect with the author's content stored on the Hive blockchain if the various frontends were no longer around. But, through experimentation with the Hive Legacy archive format I've found way more use cases than I ever imagined.
Things that I have found that can be done with a Hive Legacy archive.
An archive can be used to reconstruct a user's history on Hive. I have discovered that you can find out quite a few milestones and events in a user's history, like when a user created their account, their first post, their first front end. I have also discovered a user's frontend usage, such as when they experiment with new frontends or completely switch from one to another. And I have found when a user ends their journey on Hive. When they made their last post or comment(s), the last frontends they used, the last communities they were active in. All of that from a Hive Legacy archive.
Then there is the issue of finding a user's posts when a user's history has links from frontends in the past. With a Hive Legacy archive this can still be done since it stores the information necessary to retrieve it in the future. If someone in the future builds a tool, or uses AI, an archive could be used to retrieve the full post data from the Hive chain.
The archive can be searchable. The data in the archive can be searched for a wide variety of data points, like topics, communities or periods in time.
And this is the thing that really surprised me the most. Using an LLM to analyze a user's archive for identifying patterns or finding specific posts related to a historical event or major life experience. It amazed me how much information could be discovered from an archive that I only thought could be possible from the complete post data.
Since the format of a Hive Legacy archive is a JSON file, it can be stored most anywhere. It can be stored on a user's hard drive, a portable thumb drive or SD card, on IPFS or other file storage solutions and on archive sites like Archive.org. And if you are a little crazy, you can print it out if you have plenty of ink and paper.
This was what Hive Legacy is made for. A format to preserve a Hive user's history on the blockchain in hopes that they and their content won't be lost. That people in the future can get a picture of that Hive user by their archived history.
Generate and Enrich processes of Hive Legacy
During the development of Hive Legacy the original thought was to create the archive in one pass. We quickly found out that wasn't the best way to do it. We realized that an archive could be created with the necessary data to find a user's history on Hive. We call this the Generate feature of Hive Legacy. It does exactly what it sounds like, it calls the Hive API and then "generates" a downloadable Hive Legacy archive file for whatever Hive account the user submits. This archive can then be searched and analyzed to discover an account's posting history in a lightweight form.
A generated archive can then be further enhanced by a process we call Enrich. This is where a user can supply a Generated archive and have additional blockchain data added to the original archive. Information like transaction ID, block number and full post titles (long titles that are not truncated) can be added for deeper research and provenance.
Enrich is a longer process than Generate, so the reason for the 2-step process is to save time and API calls. If a user or researcher discovers they only need the information contained in the Generate archive then they can stop there and save time and resources on using Enrich. If they do need that additional information, they can plan accordingly.
Generate creates the Hive Legacy archive with the information needed for discovery and retrieval. Enrich is used when a user needs or wants extra information for their research or information needs. These are the processes that make up Hive Legacy to create the archive of a Hive user's history.
How to use Hive Legacy
Hive Legacy is an easy script to use since it is a single HTML file you can host locally on your own computer and open in your favorite browser. There is no need for Hive keys, Hive Keychain or Hive Signer since there is no transaction to broadcast, it just pulls the publicly available data from the blockchain.
The first step is to do a look up on the Hive account you would like to archive. This is the start of what we call the Generate portion of Hive Legacy.
Hive Legacy will then check the Hive API to see if the account exists. If it does, it will display:
- The number of posts and comments in the archive for the account.
- The date the account was created.
- The reputation score of the account.
- An option toggle for fast or considerate request pace. (This will be explained later on.)
- A button to start the generation of an archive for the account.
Once Hive Legacy completes the Generate process it will then display:
- A graphic that the Generate process was successful and an archive was created.
- The number of posts and comment in the archive for the account.
- The amount of time it took the Generate process to create the archive.
- Buttons to download the archive JSON file and to start another look-up if you like.
Once you have your Generated Hive Legacy archive you can then do a second process that we call Enrich. This process adds additional information to an existing Hive Legacy archive that wasn't added in the Generate creation of the archive.
Here you are asked to upload an existing Hive Legacy archive you wish to enrich. Hive Legacy will only accept a Hive Legacy archive and no other file type. You will get an error message if you upload a non-archive or an invalid file.
Once you upload your archive file for enriching Hive Legacy will give you a screen that shows :
- That an existing Hive Legacy archive has been loaded and the file name of the archive.
- The account name.
- The total number of entries in the archive.
- A message if the loaded archive has been enriched or not.
- Additional information about the loaded archive showing whether transaction IDs, block numbers, and full or partial titles are present.
- A toggle to select Fast or Considerate request rate. The default for Enrich is considerate, which adds a short delay between API requests.
And just like the Generate process, upon completion, you will get a screen (similar to Generate) to download your Enriched Hive Legacy archive.
The Goal of Hive Legacy
Hive Legacy is a simple idea. The Hive blockchain already preserves the data itself. Hive Legacy preserves the pointers needed to find that data.
Frontends will change , websites will disappear and domains expire. Communities will come and go. With a Hive Legacy archive, my hope is that a Hive user's history is not just preserved, but can still be found and accessed in the future when the tools Hivers used are no longer around.
That's why I built Hive Legacy.
Where to get Hive Legacy
You can download the Hive Legacy file at my GitHub. All the documentation for Hive Legacy is there too.
If you are interested in the Hive Legacy schema the GitHub also has the hive-legacy-schema.json and SCHEMA.md files for you.
I hope that the community will find Hive Legacy useful for preserving the breadcrumbs and, therefore, the history of a Hive account. I encourage you to give Hive Legacy a try and let me know what you think. Any and all feedback is welcomed.
Thanks for reading and keep on Hiving.
Dudeontheweb
AI note: I used AI to help clean up some of the grammar, spelling and wonky sentences in this post. The ideas and content are my own.
This post was written with Hive Write v0.1
