search and replace tool

search and replace tool

[dAniel hAhler]

Hello cygwin-list,

I'm looking for a search and replace tool to replace a text portion in
a bunch (3500+) of files.

That should be an easy one.. :)


-- 
shinE!
http://www.thequod.de ICQ#152282665
GnuPG/PGP key: http://thequod.de/danielhahler.asc


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

RE: search and replace tool

[Rafael Kitover]

Not really cygwin specific, you can do something like:

perl -pi -e 's/old text/new text/' `find /where -name '*.txt'`

note those are backticks surrounding the find.

Use -pi.bak to make backup files in case you screw up. You will of course need
to know how to use regular expressions (see man perlretut). If you need to do
multiline replaces, you should probably write a script, or use some other
solution.

-- 
Rafael

>-----Original Message-----
>From: cygwin-owner@cygwin.com [mailto:cygwin-owner@cygwin.com] On Behalf Of
>dAniel hAhler
>Sent: Wednesday, February 18, 2004 12:00 PM
>To: cygwin-list
>Subject: search and replace tool
>
>Hello cygwin-list,
>
>I'm looking for a search and replace tool to replace a text portion in
>a bunch (3500+) of files.
>
>That should be an easy one.. :)

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: search and replace tool

[Larry Hall]

At 02:59 PM 2/18/2004, dAniel hAhler you wrote:
>Hello cygwin-list,
>
>I'm looking for a search and replace tool to replace a text portion in
>a bunch (3500+) of files.
>
>That should be an easy one.. :)
>
>


This isn't really Cywgin-specific.  As a result, it's off-topic for this 
list.  I will, however, point you in one direction that could help.  'sed' 
or 'awk' are good possibilities here.  Of course, you'd want to wrap
these in a script that created a new file and then replaced the original 
with the result.  This same process could be used on any platform that
supports these tools however so talking much about the details isn't really
appropriate.  There's plenty of information on the net about these utilities
if you're interested in finding out more about how they work.

HTH,


--
Larry Hall                              http://www.rfk.com
RFK Partners, Inc.                      (508) 893-9779 - RFK Office
838 Washington Street                   (508) 893-9889 - FAX
Holliston, MA 01746                     


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: search and replace tool (Attn: gawk maintainer)

[Igor Pechtchanski]

On Wed, 18 Feb 2004, Larry Hall wrote:

> At 02:59 PM 2/18/2004, dAniel hAhler you wrote:
> >Hello cygwin-list,
> >
> >I'm looking for a search and replace tool to replace a text portion in
> >a bunch (3500+) of files.
> >
> >That should be an easy one.. :)
>
> This isn't really Cywgin-specific.  As a result, it's off-topic for this
> list.  I will, however, point you in one direction that could help.  'sed'
> or 'awk' are good possibilities here.  Of course, you'd want to wrap
> these in a script that created a new file and then replaced the original
> with the result.  This same process could be used on any platform that
> supports these tools however so talking much about the details isn't really
> appropriate.  There's plenty of information on the net about these utilities
> if you're interested in finding out more about how they work.
>
> HTH,
> --
> Larry Hall

Larry, you missed 'perl'.  How could you? ;-)

FWIW, both perl and sed have the ability to edit files in-place (although
this functionality is somewhat buggy in sed, see, for example,
<http://cygwin.com/ml/cygwin/2003-11/msg00773.html>).

To make this more Cygwin-related, there is no man page for "awk" in
Cygwin.  There is one for "gawk".  Since /usr/bin/awk.exe is a symlink to
/usr/bin/gawk.exe, would it be possible to create a symlink to
/usr/share/man/man1/gawk.1 as /usr/share/man/man1/awk.1?  Corinna?
	Igor
-- 
				http://cs.nyu.edu/~pechtcha/
      |\      _,,,---,,_		pechtcha@cs.nyu.edu
ZZZzz /,`.-'`'    -.  ;-;;,_		igor@watson.ibm.com
     |,4-  ) )-,_. ,\ (  `'-'		Igor Pechtchanski, Ph.D.
    '---''(_/--'  `-'\_) fL	a.k.a JaguaR-R-R-r-r-r-.-.-.  Meow!

"I have since come to realize that being between your mentor and his route
to the bathroom is a major career booster."  -- Patrick Naughton

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: search and replace tool

[Yitzchak Scott-Thoennes]

On Wed, Feb 18, 2004 at 12:13:38PM -0800, Rafael Kitover wrote:
> Not really cygwin specific, you can do something like:
> 
> perl -pi -e 's/old text/new text/' `find /where -name '*.txt'`
> 
> note those are backticks surrounding the find.
> 
> Use -pi.bak to make backup files in case you screw up. You will of course need
> to know how to use regular expressions (see man perlretut). If you need to do
> multiline replaces, you should probably write a script, or use some other
> solution.

Under cygwin, .bak is the default.  From perl's README.cygwin:
   Inplace editing C<perl -i> of files doesn't work without doing a backup
   of the file being edited C<perl -i.bak> because of windowish restrictions,
   therefore Perl adds the suffix C<.bak> automatically if you use C<perl -i>
   without specifying a backup extension.

Other options are sed or awk.

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: search and replace tool (Attn: gawk maintainer)

[Larry Hall]

At 04:46 PM 2/18/2004, Igor Pechtchanski you wrote:
>On Wed, 18 Feb 2004, Larry Hall wrote:
>
>> At 02:59 PM 2/18/2004, dAniel hAhler you wrote:
>> >Hello cygwin-list,
>> >
>> >I'm looking for a search and replace tool to replace a text portion in
>> >a bunch (3500+) of files.
>> >
>> >That should be an easy one.. :)
>>
>> This isn't really Cywgin-specific.  As a result, it's off-topic for this
>> list.  I will, however, point you in one direction that could help.  'sed'
>> or 'awk' are good possibilities here.  


<snip>


>Larry, you missed 'perl'.  How could you? ;-)


Just trying to instigate a riot. ;-)


--
Larry Hall                              http://www.rfk.com
RFK Partners, Inc.                      (508) 893-9779 - RFK Office
838 Washington Street                   (508) 893-9889 - FAX
Holliston, MA 01746                     


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: search and replace tool

[Brian Dessent]

Rafael Kitover wrote:
> 
> Not really cygwin specific, you can do something like:
> 
> perl -pi -e 's/old text/new text/' `find /where -name '*.txt'`
> 
> note those are backticks surrounding the find.
> 
> Use -pi.bak to make backup files in case you screw up. You will of course need
> to know how to use regular expressions (see man perlretut). If you need to do
> multiline replaces, you should probably write a script, or use some other
> solution.

And if you're expecting a LOT of files and/or files with odd characters
(e.g. space, quote) in their filenames, try this variant:

find /where -name \*.txt -print0 | xargs -0 perl -pe 's/old/new/`

Brian

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: search and replace tool

[Reini Urban]

Larry Hall schrieb:
> At 02:59 PM 2/18/2004, dAniel hAhler you wrote:
>>I'm looking for a search and replace tool to replace a text portion in
>>a bunch (3500+) of files.
>>
>>That should be an easy one.. :)
> 
> This isn't really Cywgin-specific.  As a result, it's off-topic for this 
> list.  

But fixing perl's long-standing inability to do direct inline editing 
via perl -i would be cygwin specific.
Anyone investigated this lately?
-- 
Reini Urban
http://xarch.tu-graz.ac.at/home/rurban/


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: search and replace tool

[Igor Pechtchanski]

On Fri, 20 Feb 2004, Reini Urban wrote:

> Larry Hall schrieb:
> > At 02:59 PM 2/18/2004, dAniel hAhler you wrote:
> >>I'm looking for a search and replace tool to replace a text portion in
> >>a bunch (3500+) of files.
> >>
> >>That should be an easy one.. :)
> >
> > This isn't really Cywgin-specific.  As a result, it's off-topic for this
> > list.
>
> But fixing perl's long-standing inability to do direct inline editing
> via perl -i would be cygwin specific.
> Anyone investigated this lately?

What on Earth are you talking about?  What inability?  WFM (see below).
	Igor

$ cygcheck -cd perl
Cygwin Package Information
Package              Version
perl                 5.8.2-1
$ ls
$ cat <<EOF > sometext
> blah
> blah and more blah
> still blah
> ok, done with blah
> EOF
$ ls
sometext
$ cat sometext
blah
blah and more blah
still blah
ok, done with blah
$ perl -i -pe 's/blah/stuff/g' sometext
$ ls
sometext  sometext.bak
$ cat sometext
stuff
stuff and more stuff
still stuff
ok, done with stuff
$

-- 
				http://cs.nyu.edu/~pechtcha/
      |\      _,,,---,,_		pechtcha@cs.nyu.edu
ZZZzz /,`.-'`'    -.  ;-;;,_		igor@watson.ibm.com
     |,4-  ) )-,_. ,\ (  `'-'		Igor Pechtchanski, Ph.D.
    '---''(_/--'  `-'\_) fL	a.k.a JaguaR-R-R-r-r-r-.-.-.  Meow!

"I have since come to realize that being between your mentor and his route
to the bathroom is a major career booster."  -- Patrick Naughton

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: search and replace tool

[Yitzchak Scott-Thoennes]

On Fri, Feb 20, 2004 at 12:47:45AM +0100, Reini Urban <rurban@x-ray.at> wrote:
> Larry Hall schrieb:
> >At 02:59 PM 2/18/2004, dAniel hAhler you wrote:
> >>I'm looking for a search and replace tool to replace a text portion in
> >>a bunch (3500+) of files.
> >>
> >>That should be an easy one.. :)
> >
> >This isn't really Cywgin-specific.  As a result, it's off-topic for this 
> >list.  
> 
> But fixing perl's long-standing inability to do direct inline editing 
> via perl -i would be cygwin specific.
> Anyone investigated this lately?

That's just a matter of allowing unlinking an open file and creating a
new one with the same name...

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: search and replace tool

[Brian Dessent]

Igor Pechtchanski wrote:

> On Fri, 20 Feb 2004, Reini Urban wrote:
>
> > But fixing perl's long-standing inability to do direct inline editing
> > via perl -i would be cygwin specific.
> > Anyone investigated this lately?
> 
> What on Earth are you talking about?  What inability?  WFM (see below).
> ...
> $ perl -i -pe 's/blah/stuff/g' sometext
> $ ls
> sometext  sometext.bak

It didn't do the editing "inline", it created a new file and renamed the
old one ".bak".  In other words, on Cygwin "-i" is really "-i.bak".  If
you try the above sequence on linux you don't get a .bak file and the
changes are truly done in-place.  I assume this relates to differences
in filesystem semantics.

Brian

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: search and replace tool

[Igor Pechtchanski]

On Thu, 19 Feb 2004, Brian Dessent wrote:

> Igor Pechtchanski wrote:
>
> > On Fri, 20 Feb 2004, Reini Urban wrote:
> >
> > > But fixing perl's long-standing inability to do direct inline editing
> > > via perl -i would be cygwin specific.
> > > Anyone investigated this lately?
> >
> > What on Earth are you talking about?  What inability?  WFM (see below).
> > ...
> > $ perl -i -pe 's/blah/stuff/g' sometext
> > $ ls
> > sometext  sometext.bak
>
> It didn't do the editing "inline", it created a new file and renamed the
> old one ".bak".  In other words, on Cygwin "-i" is really "-i.bak".  If
> you try the above sequence on linux you don't get a .bak file and the
> changes are truly done in-place.  I assume this relates to differences
> in filesystem semantics.
>
> Brian

I stand corrected.  Thanks for the explanation.
	Igor
-- 
				http://cs.nyu.edu/~pechtcha/
      |\      _,,,---,,_		pechtcha@cs.nyu.edu
ZZZzz /,`.-'`'    -.  ;-;;,_		igor@watson.ibm.com
     |,4-  ) )-,_. ,\ (  `'-'		Igor Pechtchanski, Ph.D.
    '---''(_/--'  `-'\_) fL	a.k.a JaguaR-R-R-r-r-r-.-.-.  Meow!

"I have since come to realize that being between your mentor and his route
to the bathroom is a major career booster."  -- Patrick Naughton

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: search and replace tool

[Larry Hall]

At 06:47 PM 2/19/2004, Reini Urban you wrote:
>Larry Hall schrieb:
>>At 02:59 PM 2/18/2004, dAniel hAhler you wrote:
>>>I'm looking for a search and replace tool to replace a text portion in
>>>a bunch (3500+) of files.
>>>
>>>That should be an easy one.. :)
>>This isn't really Cywgin-specific.  As a result, it's off-topic for this list.  
>
>But fixing perl's long-standing inability to do direct inline editing via perl -i would be cygwin specific.
>Anyone investigated this lately?


Huh.  Interesting way to open a new subject.  Since the text you 
quoted above makes absolutely no mention whatsoever about Perl but later
responses to this thread do, I'm going to assume that you at least read
further along, saw the subsequent suggestions about using Perl for the 
above, and then decided to reply to my post, which doesn't mention Perl,
bringing up the new topic of how -i does or doesn't work with Cygwin's
Perl.  I guess this just goes to show there are a thousand different ways
to look at things.  I never would have imagined this would seem like a 
straight line to anyone.

Anyway, as mentioned later in the thread, the problem is the implied 
semantics of "unlink()" which isn't supported by the Windows API.  So 
this long-standing inability for Perl is likely to remain for the 
foreseeable future AFAICS.

This list prefers that you start new threads for new subjects, just for
future reference.


--
Larry Hall                              http://www.rfk.com
RFK Partners, Inc.                      (508) 893-9779 - RFK Office
838 Washington Street                   (508) 893-9889 - FAX
Holliston, MA 01746                     


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: search and replace tool

[Egor Duda]

> On Thu, 19 Feb 2004, Brian Dessent wrote:
>>>What on Earth are you talking about?  What inability?  WFM (see below).
>>>...
>>>$ perl -i -pe 's/blah/stuff/g' sometext
>>>$ ls
>>>sometext  sometext.bak
>>
>>It didn't do the editing "inline", it created a new file and renamed the
>>old one ".bak".  In other words, on Cygwin "-i" is really "-i.bak".  If
>>you try the above sequence on linux you don't get a .bak file and the
>>changes are truly done in-place.  I assume this relates to differences
>>in filesystem semantics.

huh? what do you mean "in-place"? linux writes new file to new place, it 
just deletes .bak file afterwards, unlike cygwin.

deo@paltus:~$ echo aaa >xxx
deo@paltus:~$ ls -i xxx
  408096 xxx
deo@paltus:~$ perl -i -pe 's/aaa/bbbb/' xxx
deo@paltus:~$ ls -i xxx
  408074 xxx
deo@paltus:~$ cat xxx
bbbb

egor.

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: search and replace tool

[Yitzchak Scott-Thoennes]

On Fri, Feb 20, 2004 at 10:02:03AM +0300, Egor Duda wrote:
> huh? what do you mean "in-place"? linux writes new file to new place, it 
> just deletes .bak file afterwards, unlike cygwin.
> 
> deo@paltus:~$ echo aaa >xxx
> deo@paltus:~$ ls -i xxx
>  408096 xxx
> deo@paltus:~$ perl -i -pe 's/aaa/bbbb/' xxx
> deo@paltus:~$ ls -i xxx
>  408074 xxx
> deo@paltus:~$ cat xxx
> bbbb

That doesn't ever create a backup file or a temporary file.  It opens
xxx for read, unlinks it, opens xxx for writing, then reads from the
original handle and writes to the second handle.  This is AFAIUI
impossible on windows, but possible on things like unix and VMS (where
the unlink is skipped because of the automatic versioning).

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: search and replace tool

[Brian Dessent]

Egor Duda wrote:

> huh? what do you mean "in-place"? linux writes new file to new place, it
> just deletes .bak file afterwards, unlike cygwin.

Okay, it's not technically in-place.  But perl expects to be able to do
the following if you specify "-i":

x = open("file");
unlink("file");
y = open("file");
read(x, ...);
write(y, ...);

When the program ends or the filehandles are closed, there's no further
action required, such as removing the backup copy, because it's already
been unlinked.

Unlinking an open file is not supported by the windows filesystem (?) so
this sequence is impossible.  To emulate this with Cygwin, new code
would have to be added that gets called after fh 'x' is closed to remove
the backup copy.  I'm no expert on perl internals but it sounds like
this has never been done because it's nontrivial.

That's all I know.  I think Gerrit would be able to comment further on
this.

Brian

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: search and replace tool

[Egor Duda]

Yitzchak Scott-Thoennes wrote:
> On Fri, Feb 20, 2004 at 10:02:03AM +0300, Egor Duda wrote:
> 
>>huh? what do you mean "in-place"? linux writes new file to new place, it 
>>just deletes .bak file afterwards, unlike cygwin.
>>
>>deo@paltus:~$ echo aaa >xxx
>>deo@paltus:~$ ls -i xxx
>> 408096 xxx
>>deo@paltus:~$ perl -i -pe 's/aaa/bbbb/' xxx
>>deo@paltus:~$ ls -i xxx
>> 408074 xxx
>>deo@paltus:~$ cat xxx
>>bbbb
> 
> 
> That doesn't ever create a backup file or a temporary file.  It opens
> xxx for read, unlinks it, opens xxx for writing, then reads from the
> original handle and writes to the second handle.  This is AFAIUI
> impossible on windows, but possible on things like unix and VMS (where
> the unlink is skipped because of the automatic versioning).

Well, it creates new file, while old one is still present on disk. Space 
occupied by original file is freed only after handle is closed. What is 
deleted immediatly on linux is just directory entry, not the file itself.

Depending on the goal you wish to achive, cygwin may have drawbacks 
compared to linux or they may be equal.

If you want to search-and-replace in 700Mb file on 1Gb disk then you 
won't be able to do so either on linux or on cygwin.
If you wish to keep your real .bak file from being overwritten by 
search-and-replace script, then linux is slightly more convenient here, 
but on cygwin it's almost as easy to work around.

egor.

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/