Windows Illegal Character Help

Bulk Rename Utility How-To's

Re: Windows Illegal Character Help

Postby RegexNinja » Mon Apr 27, 2020 5:42 pm

Hi.. Firstly, I'm no expert on this kinda stuff, but I think support for conditionals just depends on your regex version.
Conditionals just let you test some-other-group's match to determine what another-group should match like:

(?(TestConditionOfAnotherGroup#)TrueMatch|FalseMatch)
You can reference actual Group#'s or spec them as relative, like -1 for PreviousGroup (some flavors even let you name the groups).
And you're not limited to just 1 test-condition, so you can: (True1|True2|etc)|(False1|False2|etc).
Hope it helps.. Cheers!
RegexNinja
 
Posts: 134
Joined: Fri Feb 21, 2020 5:26 pm

Re: Windows Illegal Character Help

Postby Admin » Tue Apr 28, 2020 2:59 am

As per the Admin removing the log in info, if that's the rule no problem. I created the FTP server just for this forum to assist in the troubleshooting.


Not a problem at all, just removed it for your security, but if you created it just for this example, it does not matter :) cheers
Admin
Site Admin
 
Posts: 2354
Joined: Tue Mar 08, 2005 8:39 pm

Re: Windows Illegal Character Help

Postby trm2 » Tue Apr 28, 2020 9:30 am

RegEx Ninja there is no real safe way to do it. I think you have explored all possible ways including what you did to your system.

Just to summarize though - the only three ways are to do it that I know of (unsafe):

On a separate drive - flash drive for example, place your files to rename.

1. Boot into Linux - mount drive, rename files - go back to windows
2. Run Cygwin. use the Touch command:
touch \|
touch \"
touch \>
Go back to Windows

3. Sector Editor
although you know more about this method than I - hehe

Use a separate computer that you really don't care about and do any of the above. A Test environment really calls for that.

In conclusion I don't think there is any other way but to do it outside of Windows whether it is arguably the Windows API or NTFS
that prevents you from doing it from within Windows.

Hope that helps??
trm2
 
Posts: 156
Joined: Wed Jan 15, 2020 12:47 pm

Batch remove trailing spaces from FolderNames

Postby RegexNinja » Fri May 01, 2020 11:19 am

Test Batch
@echo off
:: Intentions---- Recursively find all DirNames ending with Space, to remove all-trailing-spaces
:: Dependency1: Short-Name generation must be enabled (fsutil behavior set disable8dot3 0)
:: Dependency2: Sed.exe (& its dependencies) must reside together somewhere within %path%
::
(echo.Finding bad DirNames below %CD% Please wait...)&echo.
For /f "delims=" %%A IN ('dir/b/s/ad "* *"^|sed -n "/ $/p"^|sed -n "{G;h};$p"') DO (
For /f "delims=" %%B IN ('echo "%%A"^|sed "s/.*\\\(.*\)\x22/\1/"') DO (
For /f "delims=" %%C IN ('dir/x/ad "%%~dpnA*"^|sed -n "/<DIR>/{/ %%B$/s/.*<DIR> *\([^ ]\{7,8\}\).*/\1/p}"') DO (
@echo Ren "%%~dpA%%C\" "%%~nA"
)))
echo.Done
pause>nul


CliffyBoy
Sorry it took so long! Looked high & low for a console-app that doesnt need the Windows API.
I falsely assumed commandline-versions of WinRar/7Zip/etc would be more like their GUI counterparts.
If you know of any, it would make fixing your FolderNames easy.. Still not giving up hope..

Ironically, its simple fixing filenames like: "BadName.txt " (or "BadName " without an extension).
Since the namespace-format supports either: \\?\C:\DirPath\File and \\?\UNC\Server\Volume\File
But the trailing-text must always be a file, never a directory.. Tried everything.. Theory1 shot-down..

The DOS namespace-format: \\.\PathTo\File works the same way.. Files-only.. Theory2 shot down..
I say shot-down, maybe not, since MD 'converts' \\?\C:\DirPath\File into \\?\C:\DirPath\Folder.
I suspect this is because a FolderPath is being created, not accessed??

Any attempts to access \\?\C:\DirPath\File as \\?\C:\DirPath\BadFolderName always fails.
That's just my theory on the creation-bit, I truly have no idea... Therube, are you listening??
Maybe a separate API handles the creation of PathNames?? Again, no idea.. Neither method worked though.

At that point, I resorted to passing the folder's short-name (not always enabled) to the API.
I did write a batch to automate it, but please understand: I have a very limited test-environment.
It requires a small (80kb) commandline-app called sed to provide the filtering/regex-functionality.

You can find it at:https://sourceforge.net/projects/gnuwin32/files/sed/4.2.1/sed-4.2.1-bin.zip/download
Just extract the 1-file: /bin/sed.exe and place it anywhere within %path% and you're good to go.
For now, the batch only lists what your rename-commands would be IF you removed @echo.

Please run it from root, and give it some time.. It took about 40secs on my 250G drive.
Remember, its a ForLoop within a ForLoop, within a ForLoop, so give it at least a minute ot two.
Could've made it faster with delayed-expansion, but with ! anywhere in path, FolderNames become untouchable.

When it finishes, please post the results, or maybe upload a text-file somewhere.
Still in the testing phase for now.. I need an environment to show the weaknesses.
It does of course, rely on short-name generation (fsutil behavior set disable8dot3 0).

While it recursively fixed all of my BadFolderNames, I'm in a very limited test-environment.
I can batch-create a decent environment once, but after that, the same environment just gets recreated.
Lol, so its not much of a test after that, unless there was some kinda error.

So please dont remove @echo for now, unless you have very thoroughly looked over all the output.
Also, you can add >>C:\PathTo\BadFolderNames.txt to the end of the @echo-line to save output into a filename.
Btw, here's how the commands were 1st-listed (before executing) on my Windows system, running from root:

Ren "C:\TestDir\Illegals\Deep\Deeper\Deepest\BADDIR~1\" "BadDirNameThatHadTrailingSpaces"
Ren "C:\TestDir\Illegals\More\Deeper\Deepest\ANYNAM~2\" "AnyName-(123{!%@$&=+#&}456)-xyz"
Ren "C:\TestDir\Illegals\Deep\Deeper\BADBAD~1\" "BadBad,etc,etc"
Ren "C:\TestDir\Illegals\2AE5C~1\" "2"
Ren "C:\TestDir\Illegals\137CD~1\" "1"
Ren "C:\TMP\BADNAM~1\" "BadName1"


Some of those OrigFolderNames were like: "2 ", "1 ", etc..
Needed to test the possibilty of NoShortNameAvail, but TrailingSpace takes care of it.
To minimize errors, it lists DirNames in the order of deepest-first, then upwards towards root.

Anywho, still got some ideas, but I really need to see the output from a good environment.
If you can restore the user-account (without all the perms).. I'll post an email contact.
I wont be posting on this forum much longer, except to follow-up on any ongoing issues.

Remember, you can always use BRU's clipbard-copy to save all those BadFolderNames.
Just use #12FilterMask= $ with Regex/SubFolders/Folders checked to grab them all..
If it makes you feel safer, do the >>C:\PathTo\BadFolderNames.txt thing, so you can preview it.

Then, after previewing/editing, just rename-it to a .bat before running-it as your batch-renamer.
In the meantime, I'll keep looking for a console-app that doesnt rely on the Windows API.
This is really just an ugly workaround using short-names.. Hopefully, its just temporary.
Cheers.
RegexNinja
 
Posts: 134
Joined: Fri Feb 21, 2020 5:26 pm

Re: Windows Illegal Character Help

Postby CliffyBoy » Sat May 02, 2020 5:23 pm

RegexNinja,

At this point im getting zero assistance to trblsht the server. It wont restart so I will delete the whole thing and redo it today with the same info. If you dont mind a little visual contraband I have plenty of test folders for you to play with. I will put them in the BRU_Test folder. This should be done by 5pm today (4:38hours from now)
CliffyBoy
 
Posts: 24
Joined: Sun Apr 19, 2020 9:36 pm

Re: Windows Illegal Character Help

Postby CliffyBoy » Sat May 02, 2020 5:34 pm

RegexNinja,

In terms of a console app, Far Manager might be good for you. So far I can locate trailing space folders using the below info, but after that I'm stuck. Maybe you can figure it out and let me know. Also, the plugin LuaFAR Search is supposed to be able to do it even easier, but I cant get it to work. You much smarter than me so I think you can decipher it then let me know where im going wrong. Whatever engine they're using might be possible for BRU to utilize to get the same functionality. Check it out.

AltF7, Search for: /.+\s$/ After the search is complete, press [ Panel ] button to move all results to a temporary panel to "delete/correct" or do whatever you need
CliffyBoy
 
Posts: 24
Joined: Sun Apr 19, 2020 9:36 pm

Re: Windows Illegal Character Help

Postby CliffyBoy » Sat May 02, 2020 11:21 pm

Bad news Im behind schedule with FTP server rebuild. Good news is I located a complete solution (free). I will post back in a little while with the complete update.
CliffyBoy
 
Posts: 24
Joined: Sun Apr 19, 2020 9:36 pm

Re: Windows Illegal Character Help

Postby CliffyBoy » Sun May 03, 2020 2:04 am

RegexNinja,

I tried to send you a PM but its stuck in the outbox over there. Shmuel is the creator of the plugin LuaFarSearch. It's probably best to PM him directly with your quetions. But it does work to solve the trailing space issue. I will post full directions over here when the server is fixed.
CliffyBoy
 
Posts: 24
Joined: Sun Apr 19, 2020 9:36 pm

Re: Windows Illegal Character Help

Postby CliffyBoy » Sun May 03, 2020 2:53 pm

Final Summary Of A Solution To Remove Illegal Trailing Spaces From Files/Folders In Windows:

Install FarManager (Free)https://farmanager.com/download.php?l=en

Plugin Installation For LuaFAR Search:
1. go to https://plugring.farmanager.com/plugin.php?pid=858&l=en

2. Download either x86 or x64 version - it must match your Far Manager bitness
P.S. Do not jump from this page to Github, use one of the "Download" buttons from this page only

3. go to your Far Manager installation location, (example: C:\Users\UserName\AppData\Local\Far Manager x64\Plugins) enter Plugins folder, make a subfolder called "LuafarSearch" and unpack the plugin archive into this folder. Then restart Far Manager. Make sure the plugin's bitness matches FarManager bitness (either x86 or x64).



Search and Replace using LuaFAR Search

1. Be careful with automatic renaming - it is easy to get it wrong.

2. Using LuaFAR Search plugin: press F11, select LuaFar Search, select Rename,
Code: Select all
????????????????????????????? LF Rename ??????????????????????????????
? File mask: *                                                      ??
? (•) Search in all                 [x] Rename files                 ?
? ( ) Search in selected            [x] Rename folders               ?
? [x] Process subfolders                                             ?
??????????????????????????????????????????????????????????????????????
? Search for:                                                        ?
? \s+$                                                              ??
? Replace with:                                                      ?
?                                                                   ??
?   [ ] Function mode               [x] Confirm renaming             ?
?   [x] Log file                                                     ?




Alternate Methods:

1. There are several plugins for files/directory renaming. You can find them by searching on thishttps://forum.farmanager.com/ andhttps://plugring.farmanager.com/.

2. Press AltF7
Search for: /.+\s$/
After the search is complete, press [ Panel ] button to move all results to a temporary panel to "delete/correct" or do whatever you need

3. Alternate plugin to try: Visual Rename. It can rename files by RegExp too https://plugring.farmanager.com/



Information Helpful to BRU New Functionality for Finding and Dealing With Illegal Characters and File/Folder Trailing Spaces:

According to Pepak in the forumhttps://forum.farmanager.com/viewtopic.php?t=12123

- Archiving-apps like WinRar, 7Zip, etc do use Windows APIs. Note that there are multiple APIs that can do "the same thing", but some of them allow more detailed control over what is happening

- An app relying on the wrong API might fail to access other 'illegal' situations like "FolderNames \" that end with a trailing space.

- If anyone wants to create "BadFolderNames " for testing just press F7 and create such a directory in FAR.


**My sincere thanks to all who contributed comments and suggestions toward reaching a resolution of this dreaded windows file/folder trailing space issue.**


Note: The FTP Server is functional again and I will leave it active for a few weeks in case anyone wants to try manipulating trailing space folders without creating them in FarManager
CliffyBoy
 
Posts: 24
Joined: Sun Apr 19, 2020 9:36 pm

Re: Windows Illegal Character Help

Postby RegexNinja » Sun May 03, 2020 2:59 pm

Got the PM.. Sounds like you've had some luck.. That's fantastic news!!
Handling illegals has been a long standing issue with over here.. Seen it posted many times.

Very much looking forward to hearing this free solution.. I think many ppl will benefit from your post.
Not everyone (including me) likes dealing with batches, & its nowhere near fool-proof at this point.
Btw, does it require an installation? or can it run portably?

I try to stick with portables, but if handles them, I'd call that worthy of an install.
I thought of 1 batch-enhancement, but it only increased speed by ~20% (depends on path-structure.
Anywho, we're definitely looking forward to your post.. No rush though, know you're busy..

Edit: You beat me to it! Now seeing your post right above mine.. Cant wait to try it out!!
Cheers!
RegexNinja
 
Posts: 134
Joined: Fri Feb 21, 2020 5:26 pm

Re: Windows Illegal Character Help

Postby RegexNinja » Sun May 03, 2020 3:08 pm

CiffyBoy

I havent tried it out yet..
Can you please confirm that the plugins allow bulk-renaming of your folder-names?
Or did you have to rename them in groups? Cheers!
RegexNinja
 
Posts: 134
Joined: Fri Feb 21, 2020 5:26 pm

Re: Windows Illegal Character Help

Postby CliffyBoy » Sun May 03, 2020 3:40 pm

RegexNinja,

This program does allow bulk renaming. I was able to edit one trailing space folder at a time, then used AltF7, Search for: /.+\s$/ to locate any trailing space files/folders in all directories, then used the plugin LuaFAR Search (F11) to do the bulk renames (with or without confirmation). In my summary, I also added the comments regarding the API access which could add to BRU functionality if utilized.
CliffyBoy
 
Posts: 24
Joined: Sun Apr 19, 2020 9:36 pm

Re: Windows Illegal Character Help

Postby CliffyBoy » Sun May 03, 2020 3:43 pm

How do you edit posts? I dont see anywhere to do it.

FarManager has a portable version. https://farmanager.com/download.php?l=en
CliffyBoy
 
Posts: 24
Joined: Sun Apr 19, 2020 9:36 pm

Re: Windows Illegal Character Help

Postby RegexNinja » Sun May 03, 2020 3:54 pm

Thanks to the support over there, I can confirm this solution works 100%..
You cant beat an open-source solution.. Sorry I didnt try it earlier (was waiting on the portability question).
Which is 'yes' everybody! Its open-source and portable.. No installlation required..

TGRMN doesnt allow edits anymore.. Dont ask me why..
Again, thanks for posting this solution.. You have helped many people in doing so!
Cheers!
RegexNinja
 
Posts: 134
Joined: Fri Feb 21, 2020 5:26 pm

Re: Windows Illegal Character Help

Postby Admin » Mon May 04, 2020 2:56 am

Hi, we only allow certain validated users to edit their own posts because of spam.... :(
What happens otherwise is that spammers will submit proper "dummy" posts then later on they go back and edit their posts adding spam which we are unable to track.
So we had to disable this for all users. :(
If you want that changed for your account on the forum please let us know.
Thank you for supporting Bulk Rename Utility! 8)
Admin
Site Admin
 
Posts: 2354
Joined: Tue Mar 08, 2005 8:39 pm

PreviousNext

Return to How-To