Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 18 additions & 4 deletions interpreter/src/vm/io.rs
Original file line number Diff line number Diff line change
Expand Up @@ -58,23 +58,37 @@ impl From<&[u8]> for FilePath {
}

impl Interpreter<'_> {
pub fn begin_file_prelude(&mut self, f: Option<&Path>, res: Option<&io::Error>) {
pub fn begin_file_prelude(
&mut self,
f: Option<&Path>,
res: Option<&io::Error>,
has_begin_file: bool,
) {
let name = f.map_or(b"-".as_slice(), |p| p.as_os_str().as_encoded_bytes());
let errno = res.map_or(0, |e| e.raw_os_error().unwrap_or(-1) as _);

self.symbols.filename = Value::new_string(name.into());
self.symbols.errno = Value::new_int(errno);
self.symbols.fnr = Value::new_int(0);
self.record.clear();

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is done because inside BEGINFILE rules, we must not be able to read the last record from a previously read file. Can you confirm this behavior is not exhibited after your changes? A test would be appreciated.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, it was exhibited. With the clear dropped, BEGINFILE { print $0 } on the second file printed the last record of the first. Fixed in e989b25: begin_file_prelude clears the record again, but only when the program has a BEGINFILE rule. That is what gawk 5.3.2 does. With a BEGINFILE rule, END after an empty last file sees an empty $0. Without one, $0 survives it, which is what sylvestre's empty-file case needs.

Added beginfile_does_not_see_previous_record, checked against gawk. It fails without the change (b [last z] 2). One limit: an empty BEGINFILE {} compiles to no code, so it acts like no BEGINFILE here, the same way rule_event_loop already decides whether a program reads input.

Also rebased on main after #139; the only conflict was the two sets of tests added at the same spot. cargo test --workspace passes, clippy and fmt are clean.

// Like gawk, BEGINFILE never sees the previous file's last record.
// Without BEGINFILE, $0 survives an empty last file for END.
if has_begin_file {
self.record.clear();
}
}

// TODO: consider caching the result of all this validation, so we can
// directly dispatch to the correct functions and avoid extra writes to RT.
// TODO: on Windows, we must strip `\r` depending on BINMODE.
pub fn read_record(&mut self, reader: impl BufRead) -> Result<bool> {
pub fn read_record(&mut self, mut reader: impl BufRead) -> Result<bool> {
// At EOF, keep NR, FNR and $0 of the last record for END
if reader.fill_buf()?.is_empty() {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for the empty last file case, could we just drop the self.record.clear() in begin_file_prelude?
gawk keeps $0 until the next record is read

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 672dd5d: the clear is gone, so after an empty last file END still has $0 and NF from the last record.

One difference remains: gawk clears $0 before a BEGINFILE rule runs. gawk 'BEGINFILE{} END{print "[" $0 "]"}' a empty prints [], and this branch now prints the last line of a. Should I clear it only when the program has a BEGINFILE rule?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed on my other comment. BEGINFILE rules mustn't be able to observe previously read records.

return Ok(false);
}

// Update vars
// TODO: optimize and make more ergonomic
self.symbols.nr = &self.symbols.fnr + &Value::new_int(1);
self.symbols.nr = &self.symbols.nr + &Value::new_int(1);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What a typo lol. Thanks!

self.symbols.fnr = &self.symbols.fnr + &Value::new_int(1);

let rs = self.symbols.rs.clone();
Expand Down
12 changes: 9 additions & 3 deletions src/event.rs
Original file line number Diff line number Diff line change
Expand Up @@ -79,8 +79,9 @@ impl<'a> AwkRt<'a> {
path: Option<&Path>,
res: Option<&io::Error>,
) -> Result<bool> {
self.intrp.begin_file_prelude(path, res);
match self.drive(self.bc.begin_file_code())? {
let code = self.bc.begin_file_code();
self.intrp.begin_file_prelude(path, res, !code.is_empty());
match self.drive(code)? {
CtrlSig::End => Ok(false),
CtrlSig::NextFile => Ok(true),
CtrlSig::Exit(code) => self.end_event_loop(code).map(|_| false),
Expand All @@ -106,7 +107,12 @@ impl<'a> AwkRt<'a> {

pub fn rule_event_loop(&mut self) -> Result<()> {
let range = self.bc.rules_code();
if range.is_empty() {
// Input is read unless the program has only BEGIN rules
if range.is_empty()
&& self.bc.end_code().is_empty()
&& self.bc.begin_file_code().is_empty()
&& self.bc.end_file_code().is_empty()
{
return Ok(());
}

Expand Down
74 changes: 74 additions & 0 deletions tests/by-util/test_awk.rs
Original file line number Diff line number Diff line change
Expand Up @@ -908,3 +908,77 @@ fn slash_after_special_variable_divides() {
.succeeds()
.stdout_only("0.5 2\n1 1\n2\n3\n");
}

#[test]
fn end_sees_last_record() {
ucmd()
.arg(r"{ s += $2 } END { print NR, FNR, $0, s / NR }")
.pipe_in("a 1\nb 2\nc 6\n")
.succeeds()
.stdout_is("3 3 c 6 3\n");
}

#[test]
fn end_only_program_reads_input() {
ucmd()
.arg("END { print NR, $1 }")
.pipe_in("a\nb\n")
.succeeds()
.stdout_is("2 b\n");
}

#[test]
fn end_after_empty_file() {
let ts = uutests::util::TestScenario::new("awk");
ts.fixtures.write("a", "x y\nlast z\n");
ts.fixtures.write("empty", "");
ts.cmd(crate::TESTS_BINARY)
.args(&["END { print NR, FNR, \"[\" $0 \"]\", NF }", "empty"])
.succeeds()
.stdout_is("0 0 [] 0\n");
ts.cmd(crate::TESTS_BINARY)
.args(&["END { print NR, FNR, \"[\" $0 \"]\", NF }", "a", "empty"])
.succeeds()
.stdout_is("2 0 [last z] 2\n");
}

#[test]
fn beginfile_does_not_see_previous_record() {
let ts = uutests::util::TestScenario::new("awk");
ts.fixtures.write("a", "x y\nlast z\n");
ts.fixtures.write("b", "w\n");
ts.fixtures.write("empty", "");
ts.cmd(crate::TESTS_BINARY)
.args(&["BEGINFILE { print FILENAME, \"[\" $0 \"]\", NF }", "a", "b"])
.succeeds()
.stdout_is("a [] 0\nb [] 0\n");
// gawk clears $0 for BEGINFILE, so END after an empty last file sees it empty
ts.cmd(crate::TESTS_BINARY)
.args(&[
"BEGINFILE { n++ } END { print n, \"[\" $0 \"]\", NF }",
"a",
"empty",
])
.succeeds()
.stdout_is("2 [] 0\n");
}

#[test]
fn end_after_missing_trailing_newline() {
ucmd()
.arg(r#"END { print NR, FNR, "[" $0 "]", NF }"#)
.pipe_in("p\nq r")
.succeeds()
.stdout_is("2 2 [q r] 2\n");
}

#[test]
fn nr_counts_across_files() {
let ts = uutests::util::TestScenario::new("awk");
ts.fixtures.write("a", "x\ny\n");
ts.fixtures.write("b", "z\n");
ts.cmd(crate::TESTS_BINARY)
.args(&["{ print NR, FNR }", "a", "b"])
.succeeds()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please also add a test with an empty file and one without a trailing newline (NR/FNR in END)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added in 672dd5d: end_after_empty_file covers an empty file alone and an empty file after one with records, and end_after_missing_trailing_newline covers input with no final newline. Both check NR, FNR, $0 and NF in END, with the expected output taken from gawk 5.3.2. The empty-file test fails before the change.

.stdout_is("1 1\n2 2\n3 1\n");
}
Loading