Accidentally figured out a way to crash BRU

[howtocrashBRU]

Have been using BRU for a good while and amazingly appreciative of it. Finally learned (some) of the RegEx as well which allows for even more renaming. Using version 3.4.3.0.

Anyway, basically trying to use the Import-Rename pairs.
Done so before and it works very well once you make sure you have everything setup correctly.

Click on the usual .txt to be imported. In this case it is a UTF-16 LE file.
Load it and it crashes the entire program.

Is it because it is a UTF-16 LE file? Is there a specific character it does not like?

[howtocrashBRU]

Don't seem to be able to edit a post but tried UTF-16 BE and it also crashes. Can try the latest version next.

[howtocrashBRU]

Can set it to be UNF-8 or UNF-8 with BOM and that loads without crashing but that means it fails to handle UNF-16 correctly.

Tries to rename to something like this.

о³ÐµÑ?в¸Ñ? ³

Which is not what was intended nor what shows up on notepad!

[howtocrashBRU]

Was expecting something more like this. îâî??â???â???????

Any ideas or suggestions? Going to upgrade to the latest version and have seen the requirement to use UNF-16 but that does not seem to be fixing it here since it causes the software to crash.

[howtocrashBRU]

Well, that is also annoying. Posted that last message (which posted) but then it threw off this error. The Internet seems somewhat unhappy about not using ANSI!

General Error
SQL ERROR [ mysql4 ]

Illegal mix of collations for operation ' IN ' [1271]

An SQL error occurred while fetching this page

[howtocrashBRU]

Further testing shows that normal ANSI saved as UTF-16 characters are fine and does not cause it to crash (as expected). Once you add these not ANSI ones in and save it to UTF-16, it breaks, even if a single file rename using these characters is being requested. But they are normal UTF-16 characters!

If you save them as UTF-8, then BRU doesn't seem to treat them correctly.

All this time though, Windows is perfectly happy with them if you just manually rename. They are an actual language and should, in theory, be commonly able to be used with a regular keyboard in that country.

Using | between the filenames to be renamed (could also use ,) and if you just use the English ones, they select and will rename those as expected. Going to try to use the latest version of BRU and see if that helps any. If not then there is unfortunately some kind of minor bug unless we are missing something here. Wanted to at least ask since we could just rename these 20 or so ourselves but it was somewhat odd enough to mention and wanted to at least try to return the favor at least a little bit because BRU has been an extremely well written software and this is the first thing that has come up that has caused it to crash for us.

[howtocrashBRU]

Even more testing seems to have possibly figured out a workaround. Not entirely sure why it worked around it because we tried testing it with literally deleting all except a single not ANSI line which caused it to fail but this workaround seems to have at least made it not crash upon loading.

Not entirely sure how to paste what we saw exactly. Basically loaded it up into Calc and was double checking the rows just to make sure we had something that stood out as an issue.

Did open notepad, save a blank one to UTF-16 and then pasted a UTF-16 file and saved it. That seems to have, somewhat oddly, fixed it.

Looking more into it, it seems to be related to LF versus CR LF because once you convert a crashing one to Windows CR LF, it doesn't cause it to crash, changing nothing but converting the EOL from Unix to Windows. We can reproduce it so it seems to be some kind of bug related to UNIX LF UTF-16 LE.

Even stranger, if we make a literally almost blank, brand new, single line UNIX (LF) text file and manually put everything in and then press enter to add a new line to the end, it crashes. So maybe this isn't even related to the UTF-16 but more related to LF versus CR LF with respect to this Import Rename-Pairs issue.

Anyway, workaround for now would be to save as Windows CR LF UTF-16 LE before trying to load and run the Import Rename-Pairs.

[Admin]

Hi , thank you fore the report, could you maybe send some sample files that crash the latest version of BRU to support via e-mail please? thank you

[howtocrashBRU]

Sure. Can you quickly test by saving a file as a UNIX LF UTF-16 LE file? You will probably be able to reproduce it since it doesn't matter what is actually in the .txt file in terms of what has been added but rather it breaks as soon as you enter the last line. Not sure the code here will represent that correctly but trying to put an example below.

Code: Select all

rename.this   |   rename.that

Works fine.

Code: Select all

rename.this   |   rename.that



Breaks it. So it's not even so much that it is a UNIX LF UTF-16 LE file as much as you have the last line being a character return instead of ending there. That's most likely where the bug comes from and is likely pretty easy to fix.

[howtocrashBRU]

You can even have a literally blank .txt file that you try to import with nothing but a single UNIX character return in the entire file and it still crashes it.

Code: Select all

Seems to be the issue.

[howtocrashBRU]

Also if you remove the last UNIX \n from the very bottom of the .txt file, it prevents it from crashing at least. Almost 100% that is the issue, probably because it expects \n\r and it just gets \n. It is a Windows program after all.

\n – is the end-of-line terminator for text files in Unix
\r – was previously used as the end-of-line terminator in Mac text files, now \n is also used.
\n\r -are used to terminate lines in the DOS and Windows text files

However, if you put that \n back somewhere in the file (but NOT the last line), it then refuses to rename anything you click on which it normally would, even though it doesn't crash and it reports that the Rename Pairs are Imported. Clicking on something just doesn't convert it over as it normally would though the red Rename Pairs Imported does show up next to the 64-bit (Unicode) version as it normally does when you have Import Rename-Pairs turned on.

Removing the last basically enter key line "line" also fixes the immediate UNIX crash at least.

So the issue isn't even so much that it is UNIX versus Windows text file specifically that causes the issue but more that if you add an extra line below the "last" line, it crashes. Which in the UNIX world seems to be fairly common since this "extra line" was not added manually!

[Luuk]

The manual does say that these files should always be either ANSI or UTF-16, so with unicodes should always be the UTF-16.
Im often using UCS-2 LE BOM, but its because Im not usually have any 'surrogate' characters, and prefer the notepad++.exe.
With the notepad++.exe, you can fix the end-of-line characters with a regex-replacement like...
[\r\n]+
\r\n

[therube]

Lets talk hex .
(Cause I can't seem to get a crash.)
(Also I don't know about UTF's & UNF's...)

But, if I take a "text" file, rename-pairs.TXT:

Code: Select all

crash|me


hex'd:

Code: Select all

63 72 61 73 68 7C 6D 65 0D 0A


0D 0A, so that is Windows EOL.

If I remove OD, then I have UNIX EOL.

Code: Select all

63 72 61 73 68 7C 6D 65 0A


And if I add a "blank line", I have:

Code: Select all

63 72 61 73 68 7C 6D 65 0A 0A


or

Code: Select all

crash|me



Are some of these UNIX EOL samples supposed to cause the crash?

And you're importing & clearing the Rename-Pairs in between changing the .TXT file format?

My last attempt was something like this:

Code: Select all

FF FE 63 00 72 00 61 00 73 00 68 00 7C 00 6D 00 65 00 0D 00 0A 00 0A 0A


But still I couldn't crash.

[Luuk]

Greetings therube!
It does have to be a unicode-file for crashing, and notepad can invent them by using encoding==unicode.
A lonely 'od' or 'oa' wont terminate any of the lines, and a final lonely 'od' or 'oa' will cause the crashing.
You could just add anything like "a" afterwards to prevent the crash, but it still wont conduct properly anyways.

So the solution is to fix the end-of-line characters, but still I would prefer something more like 'illegal-format' instead of crashing.
It does seem that bru, in unicode-files, wants either 'odoa' or 'oaod' to start and end the lines, except for the 1st-line and last-line.
So then maybe a lonely end-of-line character at the very end, says begin-next-line, but then with no beginning, causes the crash???

If a file only uses 'od' or 'oa', then only the 1st-line gets conducted (because Line1 doesnt need 'odoa' to get started?).
But editing this file so that Line-3 and below use 'odoa', then Line-4 and everything below will start conducting properly.
So even though Line-3 ended with 'odoa', it wont conduct properly (because it wasnt also started with 'odoa')???

[howtocrashBRU]

And you're importing & clearing the Rename-Pairs in between changing the .TXT file format?

Yes. Including "closing" the app because it closes itself when it encounters this error. Also manually closed it after each test.

We think Luuk's example helps clarify and it seems they are reproducing this bug as well.

Looking at the literally "blank" file in hex that cause the crash, it is pretty clearly related to it being this:

FE FF 00 0A

So it starts with FE FF but then only does 0A to finish with. If it was a Windows UTF-8 text file, it would read:

0A0D and that's it.

If you then open that file then save it as UNIX UTF-16, it turns into FE FF 0A 0D 00 0D 00 0A which then crashes.

Basically it is ending in a typical UNIX way but BRU doesn't seem to quite be handling the last line properly if it is only a UNIX based 0A and nothing else. It is expecting 0A then 0D and it doesn't see it so it crashes. In a UNIX file though, the 0D is not going to be there.

Even if it either said "check your file because it has an extra UNIX line" or just assumed it was there not on purpose would be an upgrade to the entire program (and possibly everything that was entered in already) crashing. Potentially midway through a project.

If you make a text file that is literally 4 bytes (FE FF 00 0A) and save it as an actually UNIX file (or literally just those 4 bytes) and then try to import it, you will likely see the crash. We are technically one version away from the latest version but don't see anything in the changelog about it so don't think it is related but if you don't see it crash when you import that, might be missing something on our end we can look into more. Thank you for looking into this!