Windows Illegal Character Help

[therube]

Total Commander's FTP component maintained the trailing spaces on the directories - even if there were no files with the directory.
TC automatically renamed illegal characters to _'s, so "abc<>123" became abc__123.

[CliffyBoy]

Hello,

I've made some changes in the directories to address concerns. Now the structure is:
2 Directories
-Folder_Fix_Test
-Proofs

Folder_Fix_Test contains 8 subdirectories with trailing spaces, 7 of which contains no contents. 1 folder Shaft contains 3 files. The reason for this is to provide a real world example of a folder with contents that is illegally named for windows.

Proofs folder contains 13 pngs that show the currently existing trailing spaces on the BRU_Test remote server. All the files were transferred originally from the seedbox server to the windows machine where I first saw the problem, then transferred to the BRU_Test folder for public view and still retain the trailing spaces.

There are no executables. The total folder size is around 5mb. I dont know why some people are not able to transfer the trailing spaces but they are there on the server and the proofs.



RegexNinja,

I tried the BRU regex from your earlier posts and they work.

Folder-names ending with space:

#12Filters:
Mask= $
Folders/SubFolders=Checked
F5/Refresh to apply the filters

File/folder-names the begin and/or end with spaces, and/or contain: \/:?*"<>|

#12Filters:
Mask=^ | $| +\..{3,4}$|[\/:*?"<>|]
Folders/Files/SubFolders=Checked
RegEx=Checked
F5/Refresh to apply the filters


I have not been successful with this to fix the space issues. Please clarify how i should use this. Right now Im putting ^ *(.*?)[ ]*$ in section RegEx (1) in the Match text area, then putting \1 underneath in the Replace text area. I then select the Directory to search on the left Directory tree but nothing happens. Pressing Preview brings up a message saying "there are no selected items that need renaming".

This removes leading/trailing spaces from filenames, and leading spaces from folder-names:

#1Regex Match/Replace:
^ *(.*?)[ ]*$
\1


* Note- I am finding that files with leading spaces are being automatically adjusted by the Windows system once an operation is initiated on the file (moving, copying). Files I can clearly see that have leading spaces in my external drive once copied/moved to the Desktop have the leading spaces removed. It may also be renaming some files with similar numbering adding a (1) to where the space was. Im assuming it is Windows because I saw no indication that the regex filter did anything. But, for files that have legal trailing spaces, there is no change if moved or copied: examplefile .txt, movie .mp4. I don't think i have any folders with leading spaces to test if the same behavior occurs with them.

[CliffyBoy]

You guys are too fast for me! I see a whole days worth of conversation since 10am.

I did the FTP server without all the functions to make the files accessible quickly. I will work on the permissions this afternoon.

[trm2]

Shareprep can do this and more. They have trial for 50 actions or 15 days whichever comes first that you can try but it does cost $125.00 to license.
It also allows you to specify which characters you want to replace or even the replacement. It also handles long file name - (listening TGRMN??).

But again this is to remove or rename not create - and I am not about to test it with downloaded, created or otherwise invited files to find out - because unlike you people, I am not NUTS??!!

[trm2]

For those healthy minded individuals out there here is a ahk file I came across that can automatically remove illegal characters
from the clipboard.

To use:

1. Copy and paste the info into a file and name it with n AHK extension.
2. Download and install AutoHotkey
3. Click on the file you created to run it.
4. copy the bad info to the clipboard.
5. Use Ctrl-Alt_shift-G
6. Paste it back. All the illegal characters are removed.

Uses RegEx with Replace so you can easily subtract and add from the class. I made some changes to it but I also preserved the original
lines as well.

And Yes I know that it doesn't solve this poster's dilemma but it is an interesting side note just the same.

here it is -


!^+g::

; Alt(!) CTRL(^) shift(+) G ---------------- replace \? <>/\\\*""|: by _
;clipboard = ; empty clipboard

Send, {CTRLDOWN}c{CTRLUP}{ESC} ; send the ctrl C command to copy text in clipboard
ClipWait ; wait until there is something in clip bord
Removesign := RegExReplace(clipboard, "[\?<>/\\\*""|:]", "-") ; run the regex on the clipboard content and save it in Removesign
clipboard = %Removesign% ; send Removesign to clipboard

;ClipWait

return

;Tim Notes:
; I modified this to work with Clipmate.
; I rem'd out the clipboard empty because that emptied clipmate before it had a chance to removesign
; I rem'd out clipwait at the end because it interfered with clipmate - not needed.
; I modified removesign to remove the space character because it would add the replacement char '-' to every space.
; I modified the removesign to change the replacment character from underscore to hyphen
; original regex: "[\? <>/\\\*""|:]", "_")

[CliffyBoy]

I mentioned Shareprep I think twice already in this thread.

I added the Shaft folder with contents because I have noticed some different behavior when a trailing space folder has files/illegal subfolders/files in it vs not having files in it. I didnt document it at the time because I was more worried about dealing with the folder itself.

I have had files previously with other illegal characters but Filezilla catches them and adds the _ during the transfer. I dont have any right now for testing.

Write permissions are available for now as long as nobody goes crazy, and until I figure out how to set it up correctly.

[trm2]

Actually, once but whose counting:)

There is another program by the same company DelinvFile runs around $32.00 that can also handle invalid names - may want to check that out
if the other work-arounds don't do it. Not saying anything against them. I live for free work-arounds but just so that you have the options. The trial only allows
three actions. Used to be freeware for someone who may have it out there to upload that particular version.

Doesn't do file permissions or locked files - though.

If you also want to try short file names that will work with Dir /x - they have the Reg file that when run will allow Windows to use short file names.

It is called reset83.exe and I have verified that works to provide short 8.3 file names.

[CliffyBoy]

Thanks Trm2, I have DelinvFile already. Thats how I know about Shareprep. They are from the same company. DelinvFile will only delete/rename 1 file at a time. Shareprep was created to be their batch delete/renamer obstensibly for Sharepoint but applicable to other use cases as well.

[trm2]

That would be too limiting - but I believe they changed that - here is from their web page.


------------------------------
'Beginning with version 5.01, DelinvFile includes UniCode logic to enable processing of file and folder names containing characters from other languages and includes Bulk Delete, Multi-select and a new method for displaying error messages.'
Even though DelinvFile may not have handled your invalid files in the past, give the new version a try.'

-------------------------


If you have the program, see if you can get the upgrade - it is free if you have the previously paid version:


---------------------------
'
- - - - Update Policy - - - -

There are no additional charges for downloading Maintenance Releases or New Versions. There are no annual fees. '

----------------------------------

[Admin]

Hi, FYI, I removed the private login info posted on this thread as it is public accessible. cheers

[Admin]

1. While you are implementing the ability to rename illegal characters - PLEASE
add support for long files names past the 260 character limit for path\file length and implement a function,
as you have now like F8, that would allow the full viewing and editing of files. This is long overdue. It has
been asked many times. I am asking you now again.

2. Also some would like to see Global implemented in RegEx (this does not come from me),
3. direct support for Case in RegEx using something like \U and \L - I can't recall the exact Metacharacters. The non-regEx Case is fine but inadequate.
4. update the PCRE Engine if at all possible.
5. Direct support of conditionals in RegEx. Jim was working on this years ago and I believe that code still exists in the program somewhere.
6. Fix (desperately overdue) the ability to drag and drop files.

1. This is something we are looking into. In Win 10 there is the option to enable long path supports in programs, that is probably the way we will do it.https://www.tenforums.com/tutorials/51704-enable-disable-win32-long-paths-windows-10-a.html

2. Yes

3. Can you elaborate? thanks

4. OK

5. Can you give examples please?

6. Could you point to more info about this issue please? thanks

[RegexNinja]

Wow, I see I'm wayy behind here.. Cant remember the last time I saw a post this active.
Cant wait to see what-all we manage to come up with, lol.

Therube
Thanks for the tip about: MD \\?\C:\DirPath\.. Now I can quickly make 10 BadFolderNames:
For /L %A IN (1,1,10) DO @MD "\\?\C:\DirPath\BadName%A "
Works like a charm.. Thanks again!

Trm2
If you come across anything safe, that can create illegal-names on a Windows machine, please let me know.
I mean besides the 'illegal' trailing-space situation that Therube has already solved.
I'm done with hex-editors at this point, & I'm beginning to think the only answer is another drive.

Hope I'm wrong about that, but I need to create a good Windows test-environment.
I was able to create many files on the ftp server with characters like <|?*>.
But as suspected, Windows wouldnt allow FileZilla to transfer them like that.

I see you've been researching out there, so if you come across anything, please let me know.
So far, even apps with their own built-in file-handlers cant do it (within a running Windows environment).
Guess I'm a glutton for punishment, lol..

CliffyBoy
For quick reference, here's a link to the post: http://www.bulkrenameutility.co.uk/foru ... 826#p12890
Its not just for you, I need it too.. I have very old eyes, and its easy to get lost here on the forums.
But before you look at it, please note that Windows wont let BRU touch any illegals.

I should say, that since BRU uses a WindowsAPI for file-handling, it cant access the file, to rename/delete it.
Only apps with their own built-in file-handlers could access those files properly.
Technically, folder-names ending in space shouldn't be illegal, but the API certainly hates them.

That being said:
Once you manage to kill all of the illegal character situations.
This removes leading/trailing spaces from filenames, and leading spaces from folder-names:
#1Regex Match/Replace:
^ *(.*?)[ ]*$
\1

IF you're still trying to fix folder-names with trailing spaces, BRU simply cant touch them with a Window's API.
Otherwise, the regex works as described. I just used it to recursively list & remove leading/trailing spaces.
Maybe its because no files are selected in the right-pane??

By default, BRU wont auto-select anything.. You have to do it manually in the right pane.
But you can change that setting at: DisplayOptions / List / Auto-SelectEntries (v3.0.0.1).
With newer versions, its something like: Actions / List / Auto-Select.

And yes, you're correct.. Any file-operation whatsover causes the API to remove leading spaces.
But there's another API that controls the listing (not accessing) of file/folder-names.
That's why regex can list such file/folders, but not access them (rename/delete/move/etc).

The reason why 'examplefile .txt' is allowed, is because technically, it doesnt end with a space.
If you removed that extension, the API would indeed treat it, just like it treats: 'FolderName '.

How did you generate: 'no selected items'? My version doesnt have that preview-button/function?
I couldnt find it anyways, I mostly rename with an older portable version, so I'm not used to it..
Anywho, I'll wait to hear back from you on this, I still havent done any testing.. Been kinda busy.

Thanks to you/Therube, I've got some theories on how to fix such folder-names within Windows.
If that's still an issue; please let us know.. Just give me some time for testing..
And thanks for the ftp, too.. Nice app btw.

Cheers to all!

[CliffyBoy]

RegexNinja,

I was able to get the regex to work now. I thought BRU was selecting the files automatically but it wasnt. Once I selected them and entered the formula, it did fix the ones it is allowed to.

No selected items message was generated with the preview button in version 3.3.1.0. after i selected the drive in the top left pane but didnt manually selected anything in the top right pane.

If you come up with a way to bulk fix trailing space folders in Windows I am still interested.

While trying to tighten the permissions on the FTP server I think I just broke it. If I can fix it feel free to continue using it for your tests.

[trm2]

Admin.

I posted my login info? How the heck did that happen? Please elaborate so I won't do it again. I didn't see it on my end.


1. Great. I already have had my Windows set to supporting that - but it doesn't apply to third party programs.

2. Thank you.

3. PCRE2 Extended supports the use of directly using a metacharacter to force an upper or lowercase conversion in a RegEx, using \u \l \U and \L
reference -https://www.regular-expressions.info/refreplacecase.html

4. Yes! Thank you.

5. Withdrawn

6. Page 14 of the manual - 'Drag and Drop from Explorer' details this problem. it has been ongoing over many years - when you drag and drop multiple
file into BRU's Content Pane - it is VERY hit or miss. It has been known and acknowledged by many people over the years in the forum as well - and TGRMN is aware of it as I have had several discussions with them on
this topic in recent months.

Finally, I agree with RegEx Ninja that it would be great to have the option to not get locked into the current Order of Evaluation that Section 3 must be done before section 5 or section 2 cannot be run after section 4 just to throw out some examples. But that would be a major overhaul if I am not mistaken.

[CliffyBoy]

As per the Admin removing the log in info, if that's the rule no problem. I created the FTP server just for this forum to assist in the troubleshooting.