man/man1/rcalc.1

.\" RCSid "$Id: rcalc.1,v 1.16 2022/03/11 22:50:13 greg Exp $"
.TH RCALC 1 4/6/99 RADIANCE
.SH NAME
rcalc - record calculator
.SH SYNOPSIS
.B rcalc
[
.B \-b
][
.B \-l
][
.B "\-p | \-P"
][
.B \-n
][
.B \-w
][
.B \-u
][
.B \-tS
][
.B "\-i format"
][
.B "\-o format"
][
.B "\-in M"
][
.B "\-on M"
][
.B "\-f source"
][
.B "\-e expr"
][
.B "\-s svar=sval"
]
file ..
.SH DESCRIPTION
.I Rcalc
transforms ``records'' from each
.I file
according to the given set of literal and relational information.
By default, records are separated by newlines, and contain
numeric fields separated by tabs.
The
.I \-tS
option is used to specify an alternate tab character.
.PP
A
.I \-i format
option specifies a template for an alternate
input record format.
.I Format
is interpreted as a specification string if it contains a dollar sign '$'.
Otherwise, it is interpreted as the name of the file containing
the format specification.
In either case, if the format does not end with a newline, one will be added
automatically.
A special form of the
.I \-i
option may be followed immediately by a 'd' or an 'f' and an optional
count, which defaults to 1, indicating the number of double or float
binary values to read per record on the input file.
If the input is byte-swapped, the
.I \-iD
or
.I \-iF
options may be substituted.
If binary input is specified, no format string or file is needed.
.PP
A
.I \-o format
option specifies an alternate output record format.
It is interpreted the same as an input specification, except that
the special
.I \-od
or
.I \-of
options do not require a count, as this will be determined by the
number of output channels in the given expressions.
If byte-swapped output is desired, the
.I \-oD
or
.I \-oF
options may be substituted.
.PP
The
.I \-p
option specifies "passive mode," where characters that do not
match the input format are passed unaltered to the output.
If the
.I \-P
option is given instead, then valid input records that do not
yield a positive
.I cond
value are similarly passed to the output.
(See paragraph below.)\0
These options options require that the
.I \-i
input format and
.I \-o
output format also be present.
With both input and output formats, the passive mode can
effectively substitute information in the middle of a file
or stream without affecting the rest of the data.
.PP
If a
.I \-in
option is given with a positive integer argument,
.I rcalc
will stop processing input once it has loaded this number of records.
Similarly, if a
.I \-on
option is present,
.I rcalc
will stop after producing this many records, which may be a smaller
number if the
.I cond
variable is used.
If multiple input files are given, these counts are continuous over
the input and do not reset on each file.
.PP
The variable and function definitions in each
.I \-f source
file are read and compiled from the RADIANCE library where it is found.
The
.I \-e expr
option can be used to define variables on the command line.
Since many of the characters in an expression have special meaning
to the shell, it should usually be enclosed in single quotes.
The
.I \-s svar=sval
option can be used to assign a string variable a string value.
If this string variable appears in an input format, only records
with the specified value will be processed.
.PP
The
.I \-b
option instructs the program to accept only exact matches.
By default, tabs and spaces are ignored except
as field separators.
The
.I \-l
option instructs the program to ignore newlines in the input,
basically treating them the same as tabs and spaces.
Normally, the beginning of the input format matches the beginning of
a line, and the end of the format matches the end of a line.
With the
.I \-l
option, the input format can match anywhere on a line.
.PP
The
.I \-w
option causes non-fatal error messages (such as division by zero) to
be supressed.
The
.I \-u
option causes output to be flushed after each record.
The
.I \-n
option tells the program not to get any input, but to produce a
single output record.
Otherwise, if no files are given, the standard input is read.
.PP
Format files associate names with string and numeric fields
separated by literal information in a record.
A numeric field is given in a format file as a dollar sign, followed
by curly braces enclosing a variable name:
.PP
        This is a numeric field: ${vname}
.PP
A string variable is enclosed in parentheses:
.PP
        This is a string field: $(sname)
.PP
The program attempts to match literal information in
the input format to its input and assign string and numeric
fields accordingly.
If a string or numeric field variable appears more than once in
the input format, input values for the corresponding fields must
match (ie. have the same value) for the whole record to match.
Numeric values are allowed some deviation, on the order of 0.1%, but
string variables must match exactly.
Thus, dummy variables for "don't care" fields should be given unique
names so that they are not all required to take on the same value.
.PP
For each valid input record, an output record is produced
in its corresponding format.
Output field widths are given implicitly by the space occupied
in the format file, including the dollar sign and braces.
This makes it impossible to produce fields with fewer than four
characters.
If the
.I \-b
option is specified, input records must exactly match the
template.
By default, the character following each input field is used as
a delimiter.
This implies that string fields that are followed by white space
cannot contain strings with white space.
Also, numeric fields followed but not preceded by white space
will not accept numbers preceded by white space.
Adjacent input fields are advisable only with the
.I \-b
option.
Numeric output fields may contain expressions as well as variables.
A dollar sign may appear in a literal as two dollar signs ($$).
.PP
The definitions specified in
.I \-e
and
.I \-f
options relate numeric output fields to numeric input fields.
For the default record format, a field is a variable of the form
$N, where N is the column number, beginning with 1.
Output columns appear on the left-hand side of assignments, input
columns appear on the right-hand side.
.PP
A variable definition has the form:
.PP

        var = expression ;

.PP
Any instance of the variable in an expression will be replaced
with its definition.
.PP
An expression contains real numbers, variable names, function calls,
and the following operators:
.PP
        +  -  *  /  ^
.PP
Operators are evaluated left to right, except '^',
which is right associative.
Powers have the highest precedence; multiplication and
division are evaluated before addition and subtraction.
Expressions can be grouped with parentheses.
All values are double precision real.
.PP
A function definition has the form:
.PP

        func(a1, a2, ..) = expression ;

.PP
The expression can contain instances of the function arguments
as well as other variables and functions.
Function names can be passed as arguments.
Recursive functions can be defined using calls to the defined
function or other functions calling the defined function.
.PP
The variable
.I cond,
if defined, will determine whether the current input record produces
an output record.
If
.I cond
is positive, output is produced.
If
.I cond
is less than or equal to zero, the record is skipped (or passed to the
output if
.I \-P
is specified) and no other expressions
are evaluated.
This provides a convenient method for avoiding inappropriate calculations.
The following library of pre-defined functions and variables is provided:
.TP 10n
.BR $N
Return the value for input column
.I N.
If an input format is given, using a channel number generates an error.
.TP
.BR "in(n)"
Return the value for input column
.I n,
or the number of columns available in this record if
.I n
is 0.
This is an alternate way to get a column value instead of using
the $N notation, and is more flexible since it is programmable.
This function is disabled if an input format is used.
.TP
.BR "if(test, then, else)"
if test is greater than zero,
then is evaluated, otherwise else is evaluated.
This function is necessary for recursive definitions.
.TP
.BR "select(N, a1, a2, ..)"
return aN (N is rounded to the nearest integer).
This function provides array capabilities.
If
.I N
is zero, the number of available arguments is returned.
.TP
.BR "rand(x)"
compute a random number between 0 and 1 based on x.
.TP
.BR "min(a1, a2, ..)"
return the minimum value from a list of arguments.
.TP
.BR "max(a1, a2, ..)"
return the maximum value from a list of arguments.
.TP
.BR "floor(x)"
return largest integer not greater than x.
.TP
.BR "ceil(x)"
return smallest integer not less than x.
.TP
.BR "sqrt(x)"
return square root of x.
.TP
.BR "exp(x)"
compute e to the power of x (e approx = 2.718281828).
.TP
.BR "log(x)"
compute the logarithm of x to the base e.
.TP
.BR "log10(x)"
compute the logarithm of x to the base 10.
.TP
.BR PI
the ratio of a circle's circumference to its diameter.
.TP
.BR recno
the number of records recognized thus far.
.TP
.BR outno
the number or records output thus far (including this one).
.TP
.BR "sin(x), cos(x), tan(x)"
trigonometric functions.
.TP
.BR "asin(x), acos(x), atan(x)"
inverse trigonometric functions.
.TP
.BR "atan2(y, x)"
inverse tangent of y/x (range \-pi to pi).
.SH EXAMPLE
To print the square root of column two in column one,
and column one times column three in column two:
.IP "" .2i
rcalc -e '$1=sqrt($2);$2=$1*$3' inputfile > outputfile
.SH ENVIRONMENT
RAYPATH         the directories to check for auxiliary files.
.SH AUTHOR
Greg Ward
.SH BUGS
String variables can only be used in input and output formats and
.I \-s
options, not in definitions.
.PP
Tabs count as single spaces inside fields.
.PP
The
.I \-P
option buffers up to 16 Kbytes of data per record.
Longer records passed because the "cond" variable
evaluates to <= 0 will be partial; a warning will
be generated and the string "*** MISSING DATA ***"
will appear at the break.
The only workaround is to use the
.I \-p
option instead, which does not pass rejected records.
.SH "SEE ALSO"
cnt(1), ev(1), getinfo(1), icalc(1), rcollate(1), rlam(1),
rsplit(1), tabfunc(1), total(1)
Revision:	1.17
Committed:	Sun Mar 13 16:11:48 2022 UTC (3 years, 2 months ago) by greg
Branch:	MAIN
CVS Tags:	rad5R4, HEAD
Changes since 1.16:	+12 -1 lines
Log Message:	feat(rcalc): Added warning and embedded message when buffer overflows with -P
#	User	Rev	Content
1	greg	1.17	.\" RCSid "$Id: rcalc.1,v 1.16 2022/03/11 22:50:13 greg Exp $"
2	greg	1.1	.TH RCALC 1 4/6/99 RADIANCE
3			.SH NAME
4			rcalc - record calculator
5			.SH SYNOPSIS
6			.B rcalc
7			[
8			.B \-b
9			][
10			.B \-l
11			][
12	greg	1.16	.B "\-p \| \-P"
13	greg	1.6	][
14	greg	1.1	.B \-n
15			][
16			.B \-w
17			][
18			.B \-u
19			][
20			.B \-tS
21			][
22			.B "\-i format"
23			][
24			.B "\-o format"
25			][
26	greg	1.12	.B "\-in M"
27			][
28			.B "\-on M"
29			][
30	greg	1.1	.B "\-f source"
31			][
32			.B "\-e expr"
33			][
34			.B "\-s svar=sval"
35			]
36			file ..
37			.SH DESCRIPTION
38			.I Rcalc
39			transforms ``records'' from each
40			.I file
41			according to the given set of literal and relational information.
42			By default, records are separated by newlines, and contain
43			numeric fields separated by tabs.
44			The
45			.I \-tS
46			option is used to specify an alternate tab character.
47	greg	1.2	.PP
48	greg	1.1	A
49			.I \-i format
50			option specifies a template for an alternate
51			input record format.
52			.I Format
53			is interpreted as a specification string if it contains a dollar sign '$'.
54			Otherwise, it is interpreted as the name of the file containing
55			the format specification.
56			In either case, if the format does not end with a newline, one will be added
57			automatically.
58	greg	1.2	A special form of the
59			.I \-i
60			option may be followed immediately by a 'd' or an 'f' and an optional
61			count, which defaults to 1, indicating the number of double or float
62			binary values to read per record on the input file.
63	greg	1.7	If the input is byte-swapped, the
64			.I \-iD
65			or
66			.I \-iF
67			options may be substituted.
68	greg	1.2	If binary input is specified, no format string or file is needed.
69			.PP
70	greg	1.1	A
71			.I \-o format
72			option specifies an alternate output record format.
73	greg	1.2	It is interpreted the same as an input specification, except that
74			the special
75			.I \-od
76			or
77			.I \-of
78			options do not require a count, as this will be determined by the
79			number of output channels in the given expressions.
80	greg	1.7	If byte-swapped output is desired, the
81			.I \-oD
82			or
83			.I \-oF
84			options may be substituted.
85	greg	1.2	.PP
86	greg	1.6	The
87			.I \-p
88			option specifies "passive mode," where characters that do not
89			match the input format are passed unaltered to the output.
90	greg	1.16	If the
91			.I \-P
92			option is given instead, then valid input records that do not
93			yield a positive
94			.I cond
95			value are similarly passed to the output.
96			(See paragraph below.)\0
97			These options options require that the
98	greg	1.6	.I \-i
99	greg	1.16	input format and
100	greg	1.6	.I \-o
101	greg	1.16	output format also be present.
102	greg	1.6	With both input and output formats, the passive mode can
103			effectively substitute information in the middle of a file
104			or stream without affecting the rest of the data.
105			.PP
106	greg	1.12	If a
107			.I \-in
108			option is given with a positive integer argument,
109			.I rcalc
110			will stop processing input once it has loaded this number of records.
111			Similarly, if a
112			.I \-on
113			option is present,
114			.I rcalc
115			will stop after producing this many records, which may be a smaller
116			number if the
117			.I cond
118			variable is used.
119			If multiple input files are given, these counts are continuous over
120			the input and do not reset on each file.
121			.PP
122	greg	1.1	The variable and function definitions in each
123			.I \-f source
124	greg	1.11	file are read and compiled from the RADIANCE library where it is found.
125	greg	1.1	The
126			.I \-e expr
127			option can be used to define variables on the command line.
128			Since many of the characters in an expression have special meaning
129			to the shell, it should usually be enclosed in single quotes.
130			The
131			.I \-s svar=sval
132			option can be used to assign a string variable a string value.
133			If this string variable appears in an input format, only records
134			with the specified value will be processed.
135	greg	1.2	.PP
136	greg	1.1	The
137			.I \-b
138			option instructs the program to accept only exact matches.
139			By default, tabs and spaces are ignored except
140			as field separators.
141			The
142			.I \-l
143			option instructs the program to ignore newlines in the input,
144			basically treating them the same as tabs and spaces.
145			Normally, the beginning of the input format matches the beginning of
146			a line, and the end of the format matches the end of a line.
147			With the
148			.I \-l
149			option, the input format can match anywhere on a line.
150	greg	1.2	.PP
151	greg	1.1	The
152			.I \-w
153			option causes non-fatal error messages (such as division by zero) to
154			be supressed.
155			The
156			.I \-u
157			option causes output to be flushed after each record.
158			The
159			.I \-n
160			option tells the program not to get any input, but to produce a
161			single output record.
162			Otherwise, if no files are given, the standard input is read.
163			.PP
164			Format files associate names with string and numeric fields
165			separated by literal information in a record.
166			A numeric field is given in a format file as a dollar sign, followed
167			by curly braces enclosing a variable name:
168			.PP
169			This is a numeric field: ${vname}
170			.PP
171			A string variable is enclosed in parentheses:
172			.PP
173			This is a string field: $(sname)
174			.PP
175			The program attempts to match literal information in
176			the input format to its input and assign string and numeric
177			fields accordingly.
178			If a string or numeric field variable appears more than once in
179			the input format, input values for the corresponding fields must
180			match (ie. have the same value) for the whole record to match.
181			Numeric values are allowed some deviation, on the order of 0.1%, but
182			string variables must match exactly.
183			Thus, dummy variables for "don't care" fields should be given unique
184			names so that they are not all required to take on the same value.
185			.PP
186			For each valid input record, an output record is produced
187			in its corresponding format.
188			Output field widths are given implicitly by the space occupied
189			in the format file, including the dollar sign and braces.
190			This makes it impossible to produce fields with fewer than four
191			characters.
192			If the
193			.I \-b
194			option is specified, input records must exactly match the
195			template.
196			By default, the character following each input field is used as
197			a delimiter.
198			This implies that string fields that are followed by white space
199			cannot contain strings with white space.
200			Also, numeric fields followed but not preceded by white space
201			will not accept numbers preceded by white space.
202			Adjacent input fields are advisable only with the
203			.I \-b
204			option.
205			Numeric output fields may contain expressions as well as variables.
206			A dollar sign may appear in a literal as two dollar signs ($$).
207			.PP
208			The definitions specified in
209			.I \-e
210			and
211			.I \-f
212			options relate numeric output fields to numeric input fields.
213			For the default record format, a field is a variable of the form
214			$N, where N is the column number, beginning with 1.
215			Output columns appear on the left-hand side of assignments, input
216			columns appear on the right-hand side.
217			.PP
218			A variable definition has the form:
219			.PP
220
221			var = expression ;
222
223			.PP
224			Any instance of the variable in an expression will be replaced
225			with its definition.
226			.PP
227			An expression contains real numbers, variable names, function calls,
228			and the following operators:
229			.PP
230			+ - * / ^
231			.PP
232	greg	1.9	Operators are evaluated left to right, except '^',
233			which is right associative.
234	greg	1.1	Powers have the highest precedence; multiplication and
235			division are evaluated before addition and subtraction.
236			Expressions can be grouped with parentheses.
237			All values are double precision real.
238			.PP
239			A function definition has the form:
240			.PP
241
242			func(a1, a2, ..) = expression ;
243
244			.PP
245			The expression can contain instances of the function arguments
246			as well as other variables and functions.
247			Function names can be passed as arguments.
248			Recursive functions can be defined using calls to the defined
249			function or other functions calling the defined function.
250			.PP
251			The variable
252			.I cond,
253			if defined, will determine whether the current input record produces
254			an output record.
255			If
256			.I cond
257			is positive, output is produced.
258			If
259			.I cond
260	greg	1.16	is less than or equal to zero, the record is skipped (or passed to the
261			output if
262			.I \-P
263			is specified) and no other expressions
264	greg	1.1	are evaluated.
265			This provides a convenient method for avoiding inappropriate calculations.
266			The following library of pre-defined functions and variables is provided:
267			.TP 10n
268	greg	1.13	.BR $N
269	greg	1.9	Return the value for input column
270			.I N.
271			If an input format is given, using a channel number generates an error.
272			.TP
273	greg	1.3	.BR "in(n)"
274			Return the value for input column
275			.I n,
276			or the number of columns available in this record if
277			.I n
278			is 0.
279			This is an alternate way to get a column value instead of using
280			the $N notation, and is more flexible since it is programmable.
281			This function is disabled if an input format is used.
282			.TP
283	greg	1.16	.BR "if(test, then, else)"
284			if test is greater than zero,
285	greg	1.1	then is evaluated, otherwise else is evaluated.
286			This function is necessary for recursive definitions.
287			.TP
288			.BR "select(N, a1, a2, ..)"
289			return aN (N is rounded to the nearest integer).
290			This function provides array capabilities.
291			If
292			.I N
293			is zero, the number of available arguments is returned.
294			.TP
295			.BR "rand(x)"
296			compute a random number between 0 and 1 based on x.
297			.TP
298	greg	1.14	.BR "min(a1, a2, ..)"
299			return the minimum value from a list of arguments.
300			.TP
301			.BR "max(a1, a2, ..)"
302			return the maximum value from a list of arguments.
303			.TP
304	greg	1.1	.BR "floor(x)"
305			return largest integer not greater than x.
306			.TP
307			.BR "ceil(x)"
308			return smallest integer not less than x.
309			.TP
310			.BR "sqrt(x)"
311			return square root of x.
312			.TP
313			.BR "exp(x)"
314			compute e to the power of x (e approx = 2.718281828).
315			.TP
316			.BR "log(x)"
317			compute the logarithm of x to the base e.
318			.TP
319			.BR "log10(x)"
320			compute the logarithm of x to the base 10.
321			.TP
322			.BR PI
323			the ratio of a circle's circumference to its diameter.
324			.TP
325			.BR recno
326			the number of records recognized thus far.
327			.TP
328			.BR outno
329			the number or records output thus far (including this one).
330			.TP
331			.BR "sin(x), cos(x), tan(x)"
332			trigonometric functions.
333			.TP
334			.BR "asin(x), acos(x), atan(x)"
335			inverse trigonometric functions.
336			.TP
337			.BR "atan2(y, x)"
338	greg	1.8	inverse tangent of y/x (range \-pi to pi).
339	greg	1.1	.SH EXAMPLE
340			To print the square root of column two in column one,
341			and column one times column three in column two:
342			.IP "" .2i
343	greg	1.15	rcalc -e '$1=sqrt($2);$2=$1*$3' inputfile > outputfile
344	greg	1.11	.SH ENVIRONMENT
345			RAYPATH the directories to check for auxiliary files.
346	greg	1.1	.SH AUTHOR
347			Greg Ward
348			.SH BUGS
349			String variables can only be used in input and output formats and
350			.I \-s
351			options, not in definitions.
352			.PP
353			Tabs count as single spaces inside fields.
354	greg	1.17	.PP
355			The
356			.I \-P
357			option buffers up to 16 Kbytes of data per record.
358			Longer records passed because the "cond" variable
359			evaluates to <= 0 will be partial; a warning will
360			be generated and the string "* MISSING DATA *"
361			will appear at the break.
362			The only workaround is to use the
363			.I \-p
364			option instead, which does not pass rejected records.
365	greg	1.1	.SH "SEE ALSO"
366	greg	1.15	cnt(1), ev(1), getinfo(1), icalc(1), rcollate(1), rlam(1),
367			rsplit(1), tabfunc(1), total(1)