Ksh mystery: how are newlines represented in files?

  • Thread starter Thread starter gnome
  • Start date Start date
  • Tags Tags
    files Mystery
Click For Summary
SUMMARY

The discussion centers on handling newline characters in Korn shell (ksh) variables. The user demonstrates that while a variable defined with a newline (e.g., var="abc\ndef") behaves as expected when printed, subsequent attempts to remove the newline using the pattern operator (${var//\\n/}) fail after reading from a file. The user discovers a workaround using the 'cat -E' command to visualize and subsequently remove the newline character, but seeks a more direct method to edit out the newline from the variable.

PREREQUISITES
  • Understanding of Korn shell (ksh) syntax and variable manipulation
  • Familiarity with command-line text processing tools like 'cat'
  • Knowledge of string pattern replacement in shell scripting
  • Basic file I/O operations in Unix-like systems
NEXT STEPS
  • Research Korn shell variable manipulation techniques
  • Learn about the 'cat' command and its options, particularly '-E'
  • Explore string pattern replacement methods in ksh
  • Investigate newline character representations in Unix files
USEFUL FOR

This discussion is beneficial for shell script developers, system administrators, and anyone working with text processing in Korn shell who needs to understand newline character handling and variable manipulation.

gnome
Messages
1,031
Reaction score
1
Please take a look at these lines which demonstrate my problem in a korn shell terminal:

$ var="abc\ndef"
$ print "$var"
abc
def
$ print "$var" > file1
$ var=${var//\\n/}
$ print $var
abcdef
$ var=$(<file1)
$ print $var
abc
def
$ var=${var//\\n/}
$ print $var
abc
def
$

First I defined var with a newline embedded in it. I printed it and, as expected, got
abc
def.
I printed the same thing to file1.

Next, I edited it using the pattern operator ${var//\\n\} to remove the newline, printed it and again got what I expected. Now var is
abcdef

Then, I replaced var with the contents of file1: var=$(<file1)
Printed it, and again there's
abc
def

So far, so good.
Now, I try to edit it again with exactly the same command var=${var//\\n/}
But it has no effect. Var still prints as
abc
def

What's going on here? Is the newline represented differently in file1? How? How can I edit it out?

One other observation:
I noticed that in the FIRST instance, after defining var="abc\ndef", if I entered
print $var
I got
abc def
Only by entering
print "$var"
would I get
abc
def

But after reading it back in from the file, it prints as
abc
def
whether I enter
print "$var"
or
print $var
 
Computer science news on Phys.org
Well, I found a solution using the cat -E option that gives me a way to edit out the mystery newline character without knowing exactly what it is. For example:

$ var="abc\ndef"
$ print "$var"
abc
def
$ print "$var" > file1
$ cat file1
abc
def
$ var=$(cat -E file1)
$ print "$var"
abc$
def$
$ var=${var//\$?/}
$ var=${var//\$*/}
$ print "$var"
abcdef

But if anybody knows what that character is & how to edit it out of a variable without going through this merry-go-round procedure, please let me know.
 

Similar threads

  • Sticky
  • · Replies 1 ·
Replies
1
Views
16K
  • · Replies 4 ·
Replies
4
Views
1K
  • · Replies 17 ·
Replies
17
Views
3K
  • · Replies 7 ·
Replies
7
Views
3K
Replies
3
Views
1K
  • · Replies 5 ·
Replies
5
Views
2K
  • · Replies 3 ·
Replies
3
Views
1K
  • · Replies 30 ·
2
Replies
30
Views
5K
Replies
1
Views
2K
  • · Replies 4 ·
Replies
4
Views
2K