Windows Illegal Character Help

[RegexNinja]

Cliffyboy
Thanks for making the folders avail via ftp!! I downloaded the FileZilla client v3.47.2.1 and changed:
Edit/Settings/Transfers/ Unchecked: Enable invalid character filtering (& killed _ as their replacement)
My hope was to keep as many illegal-chars as possible, if you have more illegals (<|?*>), would appreciate.

Can you please verify that all 4 folders beneath 'BRU_Test' have the trailing spaces?
The 1st 3-folders (with 'balloon') dont transfer that way, & the 4th-folder is over 400MB.
I dont have 1 of those unlimited data plans, so unfortunately, that's an issue for me.

I did manage to get the whole folder downloaded.. Thanks again for that!
Sure enough, BRU easily finds such folder-names with: Space$ in its #12Filter.
So it'd be very easy to find them all with SubFolders checked.

Also, thanks for testing the manual rename (without compressing, then extracting).
That confirms WinRar uses its own internal file-handler, versus a Windows API.
So its definitely possible to code BRU the same way.

I say possible, but again.. No idea how hard it is to code your own file-handler.
IF you have any smaller/empty folders with trailing spaces, could you please add them beneath BRU_Test?
I've many theories I wanna test out, but if they work, I'll have to download another 400MB, just for 1 folder-name.

Therube
I cant get that to work on my system. Does it require file-sharing to be enabled?
Are you able to perform those renames on the local drive? Is 'xxx' a file or folder?
I've tried both: "\\?\C:\TMP\illegal\xxx" and "\\%computername%\C:\TMP\illegal\xxx"

In my expiraments, I kept getting: 'syntax incorrect' or 'network path not found'.
Ugh.. For now, it seems the best solution for getting illegals onto my system is FTP.
Cheers!

[trm2]

Dear Admin.

Since you are considering a new version:

1. While you are implementing the ability to rename illegal characters - PLEASE
add support for long files names past the 260 character limit for path\file length and implement a function,
as you have now like F8, that would allow the full viewing and editing of files. This is long overdue. It has
been asked many times. I am asking you now again.

2. Also some would like to see Global implemented in RegEx (this does not come from me),
3. direct support for Case in RegEx using something like \U and \L - I can't recall the exact Metacharacters. The non-regEx Case is fine but inadequate.
4. update the PCRE Engine if at all possible.
5. Direct support of conditionals in RegEx. Jim was working on this years ago and I believe that code still exists in the program somewhere.
6. Fix (desperately overdue) the ability to drag and drop files.

Normally this would be posted in the suggestions forum but I think it is important to mention it here since you are referencing a new update in this post.

Regex Ninja has another wish list to add I'm sure.

[therube]

There are 4 folders; 3 with no contents just named with a trailing space, and 1 with a trailing space name with some music files. Total size around 500mb.

I guess you've changed it.
Total size ~1 MB.
And only the directory, "Shaft (1971, FLAC) - Isaac Hayes " has a trailing space.

(Trailing space'd file/dir can be created as mentioned above.)

What about other illegal characters?
Trailing space, trailing dots (.), we have & can create.

How about other illegals; <>|... ?

[therube]

rename "\\?\C:\TMP\illegal\xxx" " XYZ "

(1) C:\TMP\illegal\, is the directory name.
(2) xxx, is an existing file (or could be a directory) name.
(3) " XYZ ", is what you want the newly named file name to be.

\\?, is, oh, not sure what's it's called or whatever, but says that it's a UNC path, or something of that nature.

So the directory, C:\TMP\illegal\ must exist.
The file xxx must exist in that directory.

The rename command specifies (1)+(2) the UNC path+filename, (3) the " spaced " file/dir name is only the name itself - without path.
I executed the rename (command) from within the C:\TMP\illegal directory.
(Think that shouldn't matter so long as "correct" path & name are provided?)

[therube]

I guess you've changed it.
Total size ~1 MB.

That's weird, /files/ just turned up (& that is large).

only the directory, "Shaft (1971, FLAC) - Isaac Hayes " has a trailing space

That's also weird.
Through FileZilla, I can "see" that in fact other directories (like, 025 - Red Rose) also have trailing spaces, yet when I downloaded, only the "Shaft" directory ended up with the trailing space - none of the others?


(I'm running FileZilla in Sandboxie. I wouldn't think that should matter?
As far as FileZilla's "invalid characters" setting, seemingly it does not consider spaced [trailing or otherwise] as "invalid".)

[therube]

That's also weird.
Through FileZilla, I can "see" that in fact other directories (like, 025 - Red Rose) also have trailing spaces, yet when I downloaded, only the "Shaft" directory ended up with the trailing space - none of the others?

So it seems that (in FileZilla at least) even though a directory name may physically end with a space, " 025 - Red Rose ", if it is empty, contains nothing within, when it is transferred down, locally, on that directory name, the trailing space is dropped - even though it does exist on the server end.

As you can see, I have placed the file, "testing123.txt", into the "025 - Red Rose " directory (on your server), & with that, once I download that directory (& the associated file), the directory name is named as it is on the server, with the trailing space.

I THINK YOUR PERMISSIONS ON YOUR END ARE TOO LAX & YOU WANT TO TIGHTEN THEM UP OR BLOCK ACCESS ALTOGETHER.

[RegexNinja]

Something must be different about our systems, mine refuses to touch FolderNames (files work fine):

Ren "\\?\C:\TMP\illegal\xxx" "XYZ" ---> The syntax of the command is incorrect
Ren "\\?\C:\TMP\illegal\xxx\" XYZ ----> The syntax of the command is incorrect
Ren \\?\C:\TMP\illegal\xxx XYZ ------> The syntax of the command is incorrect
Ren \\?\C:\TMP\illegal\xxx\" "XYZ" ---> The syntax of the command is incorrect

When I try the %computername% equivalents, I just get: 'network path was not found'.
Same thing with localhost.. No big deal, I can practice with files.. THANKS for that part!!
Btw, have you had any success at getting pipe-symbols and stuff in there?
I've been trying "XY|Z" and "XY^|Z" etc.. But no luck so far..

Hmmm, I dunno what to say about those 1st 3 folders?
Maybe since theres no files inside them, somehow FolderName gets written differently??
I've no idea on that one. Problem is, how does he get a file inside them so we can test?
Cheers!

[therube]

Should be working.
What happens though is that, with a directory, it looks like the trailing (space) is truncated.

Be sure your specified (UNC) path+file/dir name is correct.
(I just checked any number of time - with a wrongly typed path, & duplicated you results.
Once I realized that, got my path correct, it worked.)

[trm2]

hehe no luck so far Regex Ninja? - ohh.. I think you had some luck.

Inside joke.

[therube]

Oh wait, now I may also have mixed up my RENAME and MOVE's.
Let me see...

Yep, sure did!

RENAME will work on directory names - but not with UNC paths.
So if you have a directory /Z/:

RENAME Z ZZZ works
as does RENAME c:\tmp\illegal\Z ZZZ

This fails: RENAME \\?\c:\tmp\illegal\Z ZZZ

---

MOVE will work with UNC directory paths, but if you specify a trailing space, it looks to get truncated.

Code: Select all

MOVE  "\\?\c:\tmp\illegal\Z"    "   ZZZ   "


(The specified renamed filename has 3 proceeding & 3 trailing spaces.)

That renames the directory name from "Z" to " ZZZ" (rather then " ZZZ ").
(The resultant filename has 3 proceeding [though it doesn't display that way in the line above] & no trailing space.
Having a file within the Z directory, so Z\testing123.txt, did not change things, locally, with MOVE, as it did with FileZilla [& download via ftp].)

[RegexNinja]

Dont ask me why, but my system wont touch FolderNames in that format:

Ren "\\?\C:\TMP" "XYZ" --------------> syntax incorrect
Ren "\\?\C:\TMP\illegal" "XYZ" ------> syntax incorrect
Ren "\\?\C:\TMP\illegal\xxx" "XYZ" --> syntax incorrect, next command:
Ren "C:\TMP\illegal\xxx" "XYZ" ------> works as expected

And yet the \\?\ format works with files everytime..
I just caught your MOVE vs REN command.. Now it all makes sense..
Its no big deal now.. I didnt realize we had write permissions on the FTP server!!
So I've just been creating the folder-names there, and then transferring to my drive.

I also created some files on the server with <|?:*> in them, but the transfer always fails!
Have you figured out a way to transfer such files? I'm guessing Windows wont allow it.
Cheers!

[therube]

GNU mkdir can create directory names with trailing spaces.

Code: Select all

mkdir.exe  "   abc   ",  creates the directory named,  "   abc", opening spaces are retained, but trailing spaces are truncated
mkdir.exe  "\\?\C:\TMP\illegal\   abc   ", works, opening & trailing spaces are maintained


(You may need to specifically type in the .exe so as to not conflict with Windows MD, MKDIR built-in commands.)


Download, coreutils-5.3.0-bin.zip.
Extract mkdir.exe from within its' /bin/ directory.


ZIP'd up & listed:

Code: Select all

7-Zip 19.00 (x86) : Copyright (c) 1999-2018 Igor Pavlov : 2019-02-21

Scanning the drive for archives:
1 file, 27718 bytes (28 KiB)

Listing archive: 0.7z

--
Path = 0.7z
Type = 7z
Physical Size = 27718
Headers Size = 409
Method = LZMA2:96k BCJ
Solid = +
Blocks = 2

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
2020-04-26 11:32:50 D....            0            0  illegal
2020-04-26 10:50:47 D....            0            0  illegal\   abc123
2020-04-26 11:30:33 D....            0            0  illegal\   Z   
2020-04-26 11:30:38 D....            0            0  illegal\   Z     
2020-04-26 11:30:41 D....            0            0  illegal\   Z         
2020-04-26 11:30:44 D....            0            0  illegal\   Z           
2020-04-26 11:30:23 D....            0            0  illegal\ Z
2020-04-26 11:16:20 D....            0            0  illegal\abc123456
2020-04-26 11:30:09 D....            0            0  illegal\Z
2020-04-26 10:50:47 ....A          337          358  illegal\   abc123\xxx
2020-04-25 12:15:18 ....A          425               illegal\   XYZ3   
2020-04-25 12:14:44 ....A          335               illegal\ XYZ
2020-04-25 12:15:18 ....A          425               illegal\ XYZ.
2020-04-26 11:32:41 ....A           74               illegal\0.7z
2020-04-25 12:14:57 ....A          379               illegal\123
2020-04-25 12:15:18 ....A          425               illegal\1XYZ
2020-04-25 12:15:18 ....A          425               illegal\abc
2020-04-25 12:15:18 ....A          425               illegal\xxx
2005-04-20 20:41:12 ....A        69120        26951  illegal\mkdir.exe
------------------- ----- ------------ ------------  ------------------------
2020-04-26 11:32:50              72370        27309  10 files, 9 folders


There are 6 directories named "Z".
Simply "Z", " Z", & the four others with varying number of opening/trailing spaces.

[therube]

Oh, so you can (create on the server) ["other" illegals].

Code: Select all

Status:  Starting download of /mnt/mpathg/ziggyja/BRU_Test/Folder_Fix_Test/025 - Red Rose /abc<>123
Error:   Failed to open "C:\OUT\BRU3\abc<>123" for writing
Error:   File transfer failed


But once you select that Transfers option you mentioned (invalid character filtering):

Code: Select all

Status:   Starting download of /mnt/mpathg/ziggyja/BRU_Test/Folder_Fix_Test/025 - Red Rose /abc<>123
Status:   File transfer successful, transferred 0 bytes in 1 second


And that created the local file, named, abc__123, replaced the (Windows) illegal characters with _'s.

So to download them & have them named, illegally, locally, on Windows - no.
But you can download them, already renamed, replacing the illegal characters, automatically.

[RegexNinja]

Therube
GNU mkdir sounds like I good solution, I'd much rather have something that works locally.
Heck, I'll just rename it to something like md2 and put it in somewhere in %path%.
Thanks for the tip!

Yeah, my goal was to get as many illegals onto my system as I could, just for testing.
Got some ideas I wanna try out, but I was hoping to get lots of names with <":?*|>
I figured in the beginning that was gonna be the case, just praying I was wrong.

At least for now, we can make illegal folders with trailing-spaces to test against!
Didnt know that was possible.. I'll just have to get another drive for the true illegals.
Thanks for all the info and testing.. Cheers!

Trm2
Yeah, I'm lucky I found a boot-cd!
Manually editing one's partition table, just to get these illegal-names, was NOT the way to go.
I think you jinxed me with that post of yours: http://www.bulkrenameutility.co.uk/foru ... 826#p12904
Lesson learned, lol

WishList
Well, I think this post is one of them, to handle illegals, but they'd need to write their own file-handler.
Considering how BRU needs to mass-rename, I'm not sure how feasible it'd be.. It'd needs lots of testing, that's for sure.
API's were meant to make such things much easier.. Things get alot more complicated when you have to write your own.

I'm very curious if any archive-users out there are familiar with how well WinRar/etc handles hundreds/thousands of files.
I saw CliffyBoy's comment about how WinRar would only convert 8-files at a time.. That was not encouraging!
Hopefully that was some file-size limitation, and not the actual number of files?
Of course that's WinRar's file-handler anyway.. Doesnt mean you cant write a better one.

How about each rename-option gets own drop-down-list to choose a number, so you could do things like..
Make Regex the 4th-step in renaming, or Numbering the 1st-step, etc.. Like how BRC works.

I'd love that global-replacement feature too, maybe in a Regex check-box somewhere?
Definitely the \U \u \L \l case-converters, alot of regexers ask how to invoke it.
But my dream come true would for a window open up when you click 'multiple-regexes'

Kinda like the javascript window, and you type-in: Regex1-Match/Replace, Regex2-Match/Replace, etc
Heck, I probably wouldnt even use BRC anymore, if it had that one feature.
Of course the more they add, the slower its gonna be.. Gotta weigh what's worthwhile.
That's about it for me.. Cheers!

[therube]

Oh, silly me, kind of... Once you somewhat understand what works or how to make things work, & what doesn't work, & why it didn't work, then you think... silly, kind of.


So, GNU mkdir.exe, you don't need that.

Windows MKDIR (MD) works just fine.
It is just that you need to do it - correctly.

I must have tried something like, MD " test ", & that "failed", as noted above.
The directory " test", did get created, but the trailing space was lost.

Once you go the UNC route, then the same as with mkdir, the trail space will be retained.

Code: Select all

MD        "\\?\C:\TMP\illegal\   ZxZxZxZ         "
mkdir.exe "\\?\C:\TMP\illegal\   ZxZxZxZ         "


Both work equally as well, retaining the (9) trailing spaces in the directory names.