Posts: 241
Threads: 20
Joined: Jun 2023
Hello, I'm wondering if anyone has tried parsing this absolute wreck of a file? Whoever dumped this really had no idea what they were doing; everything is misaligned and the text encoding seems to be a tug-of-war between ANSI and UTF-8. Normally it is quite straightforward but I have given up on this one for now, too lazy to write regex for it.
Example of what I mean:
![[Image: v9egn7.png]](https://bf.st/img/a817c7282bbe3f542e9a00ff4de014a5f8be1299/68747470733a2f2f66696c65732e636174626f782e6d6f652f763965676e372e706e67)
Formerly @God, but that username was stolen from me.
Posts: 148
Threads: 28
Joined: Jun 2023
normally i can do it, but let me take a look on the data and if i have free time.
i'll post it when i done
Posts: 241
Threads: 20
Joined: Jun 2023
(Sep 24, 2023, 08:14 AM)All3in Wrote: normally i can do it, but let me take a look on the data and if i have free time.
i'll post it when i done
Thank you, much appreciated!
Formerly @God, but that username was stolen from me.
Posts: 1,493
Threads: 69
Joined: Jun 2023
Sep 24, 2023, 08:24 PM
(This post was last modified: Sep 24, 2023, 08:25 PM by Blastoise.)
I'm using the latest version of EmEditor and have absolutely no problems with the encoding,
it is detected completely correctly as UTF-8 without a signature, why can't you do the same?
https://bf.st/Thread-InterPals-Database-Leaked-Download
I am NOT a member of any of the public Telegram chats,
everyone who impersonates me on Telegram are nonentities and scammers!
Posts: 241
Threads: 20
Joined: Jun 2023
(Sep 24, 2023, 08:24 PM)Blastoise Wrote: I'm using the latest version of EmEditor and have absolutely no problems with the encoding,
it is detected completely correctly as UTF-8 without a signature, why can't you do the same?
https://bf.st/Thread-InterPals-Database-Leaked-Download
Thanks but unfortunately the official version is the messy version which I have.
Some of the names are correctly UTF-8, others are UTF-8 encoded as CP-1252 and then decoded incorrectly as UTF-8. The inconsistency causes some names to show up as mojibake, while others are fine. Normally this wouldn't matter too much but for a site based around an international audience it's very messy.
Besides this, the columns are not at synced as shown in my screenshot, I am trying to load large databases into ElasticSearch automatically so this is important.
Formerly @God, but that username was stolen from me.
Posts: 22
Threads: 2
Joined: Dec 2023
It should have been colon-separated, at least the version I am familiar with. Can you upload the tab-separated version for me? I'll see if there's something different about it.
Posts: 1,493
Threads: 69
Joined: Jun 2023
(Dec 15, 2023, 02:44 AM)Shimmer Wrote: It should have been colon-separated, at least the version I am familiar with. Can you upload the tab-separated version for me? I'll see if there's something different about it.
This tab delimited version of the database is in the official section:
https://bf.st/Thread-InterPals-Database-Leaked-Download
I am NOT a member of any of the public Telegram chats,
everyone who impersonates me on Telegram are nonentities and scammers!
|
