Interested in advertising on Derpibooru? Click here for information!
Help fund the $15 daily operational cost of Derpibooru - support us financially!
Description
No description provided.
Help fund the $15 daily operational cost of Derpibooru - support us financially!
No description provided.
Do you have these up somewhere for download? Been meaning to look at Ponibooru stuff eventually
Awesome. That’s what I’m mainly interested in.
Ah, ok, that makes sense.
Yes.
But you do still have the HTML files and not just the SQL script derived from them, right?
Ponibooru only existed from November 2010 to June 2012 and had just shy of 200k images (I have no idea how many deleted/duplicate images it had, I wasn’t able to reliably determine that from the dump data). Ponibooru did not have a forum (Shimmie does have a forum option, but it hadn’t been activated on Ponibooru), and it had nowhere near as many users or comments Derpi has. Ponibooru was only a tiny fraction of Derpi’s size.
Ponibooru didn’t use SQLite, I did. I recall Eco (Ponibooru’s admin) talking about PostgreSQL multiple times, so that’s probably what it used.
The SQLite dump was created by parsing those 200k HTML files (twice, once in Python and once in C++11; both programs outputted the same data, this was done to ensure I hadn’t made any mistakes) and extracting all image metadata, usernames, comments, faves and some other bits and pieces, then storing it in SQL tables. It does not contain things like tag histories and public user profile pages, or the images. And one immensely annoyig (to the point it actually made me rage when I realized it) thing was that comment timestamps only had 1-day accuracy! The site did keep track of exact times, but in the HTML files you will find only the date part, no time. I don’t know why.
I also did not normalize image tags, they’re stored just as plain strings. There’s no “image_taggings” table like you can find in Derpi’s database dumps. So if you were to load that SQLite file and actually use it for something, the first thing you’d have to do is to parse those tags and create a new table for them.
So yeah, in the end, Ponibooru did fit in a really small space because the dump only contained like 50% of the relevant data. You cannot “rebuild” Ponibooru from it, but it can offer you a glimpse to the site as it was at the time it was locked up.
836 and 232 megabytes ?! And SQLite ?!?! I didn’t even know Ponibooru existed, but if it was “basically Derpibooru’s predecessor”, I’d expect it to be rivaling Derpibooru in size O.o And how could the site work with an SQLite db with probly thousands, if not hundreds of, concurrent users is beyond me; yes, I know a good deal about SQLite and MySQL by experience.
In case you couldn’t tell, I’m shocked.
Wicked. I remember you messaging me about it years ago, and have been meaning to get that off you. When I get the chance (and a stable internet connection), I’ll message you and we’ll work something out.
Yes, a ZIP file containg all image pages as HTML (almost 200k files). I wrote a script that extracted all the comments and image tags and other metadata from those HTML files, then saved it in a single searchable sqlite database. The plain ZIP file is 836 megabytes, but the sqlite archive is “only” 232 megabytes.
AFAIK, the ZIP was created by jspenguin.
I know the feel bro. With it passing, a lot of early history of the fandom passed with it…though you did once tell me that you had some form of archive of all the text from it.
you need patreon or pay membership like deviantart if want keep it
Patreon didn’t exist in 2012.
@Bexar Bandito
Also mildly irritated at OP for that tease.
Technically Derpibooru’s predecessor.
Edited because: Typo
Derpibooru lived so that we could keep sinning
tips : you need patreon or pay membership like deviantart if want keep it.
Tag editing wasn’t locked up, so people creatively abused the tags of the last few images to send goodbye messages. It’s possible those tags are still archived somewhere.
world dominationpeace on earth. ^^but really a little sad form this site :(
wait what is was old news damn it!
Edited