49813: <<<: Document newline behavior and fix optimization

2022-03-30T06:07:39Z

The =(<<

49694 + doc: Allow using empty STTY= to freeze tty for a single command

2022-01-30T17:14:54Z

Previously, doing this would just run stty with no arguments, which normally causes it to print some terminal settings to stdout.

github #82: Fix typos

2021-11-12T22:54:34Z

49353: Fix comments in sourced file.

2021-09-09T19:05:39Z

If the file was sourced from an interactive shell with INTERACTIVE_COMMENTS not set, comments were not parsed. Note there is a remaining edge case where the sourced file is in fact entered at the comment line.

49307 with doc update: POSIX_TRAPS fix.

2021-08-26T08:46:39Z

With POSIX_TRAPS set, an ignored signal stays ignored when entering a subshell.

48857: declare "volatile" all globals that may be modified by signal handlers

2021-05-17T02:51:11Z

47794: exec: run final pipeline command in a subshell in sh mode

2021-04-10T22:56:39Z

zsh typically runs the final command in a pipeline in the main shell instead of a subshell. However, POSIX specifies that all commands in a pipeline run in a subshell, but permits zsh's behavior as an extension. The default /bin/sh implementations on various Linux distros and the BSDs always use a subshell for all components of a pipeline. Since zsh may be used as /bin/sh in some cases (such as macOS Catalina), it makes sense to have the common sh behavior when emulating sh, so do that by checking for being the final item of a multi-item pipeline and creating a subshell in that case. From the comment above execpline(), we know the following: last1 is a flag that this command is the last command in a shell that is about to exit, so we can exec instead of forking. It gets passed all the way down to execcmd() which actually makes the decision. A 0 is always passed if the command is not the last in the pipeline. […] If last1 is zero but the command is at the end of a pipeline, we pass 2 down to execcmd(). So there are three cases to consider in this code: • last1 is 0, which means we are not at the end of a pipeline, in which case we should not change behavior. • last1 is 1, which means we are effectively running in a subshell, because nothing that happens due to the exec is going to affect the actual shell, since it will have been replaced. So there is nothing to do here. • last1 is 2, which means our command is at the end of the pipeline, so in sh mode we should create a subshell by forking. input is nonzero if the input to this process is a pipe that we've opened. At the end of a multi-stage pipeline, it will necessarily be nonzero. Note that several of the tests may appear bizarre, since most developers do not place useless variable assignments directly at the end of a pipeline. However, as the function tests demonstrate, there are cases where assignments may occur when a shell function is used at the end of a command. The remaining assignment tests simply test additional cases, such as the use of local, that would otherwise be untested.

Allow more scripts without #!

2021-02-16T09:16:20Z

This change modifies the zsh binary safety check surrounding execve() so it can run shell scripts having concatenated binary content. We're using the same safety check as FreeBSD /bin/sh [1]. POSIX was recently revised to require this behavior: "The input file may be of any type, but the initial portion of the file intended to be parsed according to the shell grammar (XREF to XSH 2.10.2 Shell Grammar Rules) shall consist of characters and shall not contain the NUL character. The shell shall not enforce any line length limits." "Earlier versions of this standard required that input files to the shell be text files except that line lengths were unlimited. However, that was overly restrictive in relation to the fact that shells can parse a script without a trailing newline, and in relation to a common practice of concatenating a shell script ending with an 'exit' or 'exec $command' with a binary data payload to form a single-file self-extracting archive." [2] [3] One example use case of such scripts, is the Cosmopolitan C Library [4] which configuse the GNU Linker to output a polyglot shell+binary format that runs on Linux / Mac / Windows / FreeBSD / OpenBSD. [1] https://github.com/freebsd/freebsd-src/commit/9a1cd363318b7e9e70ef6af27d1675b371c16b1a [2] http://austingroupbugs.net/view.php?id=1250 [3] http://austingroupbugs.net/view.php?id=1226#c4394 [4] https://justine.lol/cosmopolitan/index.html

46268: suppress a useless compiler warning around nice()

2020-07-28T09:46:14Z

From nice(2): To detect an error, set errno to 0 before the call, and check whether it is nonzero after nice() returns -1.

46175/0003: Fix the RM_STAR_SILENT bug from the parent commit.

2020-07-05T11:11:59Z

46026: Add CLOBBER_EMPTY option.

2020-06-09T17:07:01Z

users/24909: Don't clean up special file list too early.

2020-06-08T19:52:53Z

When running a function, remove special files used for substitution after the function has run rather than before.

45923 (with memory leak fixed, cf. 45924): zprof: Don't tally all anonymous functions as though they were a single function named "(anon)".

2020-05-28T19:40:40Z

Before: % zmodload zsh/zprof % () : % () : % zprof num calls time self name ----------------------------------------------------------------------------------- 1) 2 0.08 0.04 100.00% 0.08 0.04 100.00% (anon) After: % zmodload zsh/zprof % () : % () : % zprof num calls time self name ----------------------------------------------------------------------------------- 1) 1 0.04 0.04 50.45% 0.04 0.04 50.45% (anon) [:3] 2) 1 0.04 0.04 49.55% 0.04 0.04 49.55% (anon) [:2]

45583/0008: Add the 'function -T' syntax.

2020-03-22T02:23:53Z

Config/version.mk was bumped in the previous commit.

45583/0007: WC_FUNCDEF: Add a placeholder element.

2020-03-22T02:23:53Z

45583/0001: internal: Remove a redundant assignment.

2020-03-22T02:23:52Z

The value is overwritten five lines below, without being read in the interim.

Merge remote-tracking branch 'origin/master' into 5.9

2020-03-07T21:36:46Z

* Test/D02glob.ztst: On the "unreadable directories can be globbed (users/24619, users/24626)" test, resolve conflicts by removing the Cygwin-only skip that has been added in master, since the test is passing on this branch. This effectively reverts workers/45492. See discussion starting in workers/45504. * origin/master: unposted: Remove 'sgi', as that OpenBSD port has been discontinued. 45509: fix typos in B01cd.ztst 45490 (+45495 and a test): refactor rlimits.c github #49: Fix typo: longson should be loongson users/24710: Fix job control problem with sudo. 45492: skip test added by users/24633 on Cygwin 45488: COMP_WORDS for bash need "$@"-style quoting 45487: Missing mod_export declarations for AIX 45447: Complete vcs_info_hookadd and vcs_info_hookdel. Expose _vcs_info_hooks as a top-level helper function. 45463: test: kill: Document why we use SIGURG 45453: builtins: kill: Do not signal current process group when pid is empty 45452: builtins: kill: Add `kill ''` regression test with explicit sigspec 45451: builtins: kill: Add basic test suite github #48/0002: vcs_info git: properly detect bare repositories github #48/0001: vcs_info git: avoid warnings in bare repositories unposted: Post-release version bump unposted: Release 5.8 CVE-2019-20044: Update change log for preceding commits Update NEWS/README Add unsetopt/PRIVILEGED tests Clean up error-message white space Improve PRIVILEGED fixes (again) Improve PRIVILEGED fixes Drop privileges securely unposted: V01zmodload: Fix failing test from workers/45385 45423: _su: Improve arg handling, shell look-ups unposted: _zip: Recognise '--' 45385: Add a test for 'zmodload -Fa' preemptively disabling ("blacklisting"?) features. unposted: Test release: 5.7.1-test-3 zsh/system: Fix infinite loop in sysread _diff_options: Restore -w completion lost in workers/43351 unposted: Fix ChangeLog typo. 45368: Add tests for workers/45367's issue about double slashes in 'cd -P' and /home/daniel/in/zsh. 45373: Fix ERR_EXIT bug in else branch of if. 45372: Record a symlink loop bug involving :P 45365: _git: Fix __git_recent_branches for the case when a commit has an empty message 45343: Queue signals around arithmetic evaluations 45344: Document where third-party completion functions should be installed. 45345: internal: ztst.vim: Fix highlighting of zsh comments in test payload unposted: internal: Add some comments and fix indentation. No functional change. 45340: internal: Document the difference between paramtab and realparamtab. 45332: _git: add completion for git-version _brace_parameter: add missing \ Conflicts: ChangeLog Test/D02glob.ztst Test/V01zmodload.ztst

users/24710: Fix job control problem with sudo.

2020-02-27T20:42:16Z

If we use kill to test for continued existence of a process group, we should check on failure that the error is ESRCH, as EPERM indicates the group still has memebers but running privileged so should be left alone.

45343: Queue signals around arithmetic evaluations

2020-01-29T09:02:49Z

The queueing added in execarith() in 45083 is reverted since the callee does this now.

45131: Make a function that redefines itself preserve its tracedness.

2019-12-26T04:20:10Z

This makes it easy to apply local tracing ('functions -T') to autoloadable functions that redefines themselves when first loaded.

45083: Add signal protection to execarith().

2019-12-18T10:51:59Z

Otherwise we could get re-entrancy in memory functions when setting variables.

45066: internal: Document forklevel, locallevel, and exit_pending.

2019-12-18T06:01:50Z

45058: internal: Add symbolic names to possible values of zexit()'s "from_where" parameter. No functional change.

2019-12-17T05:12:36Z

45025: fix re-entrancy problem with memory management in readoutput().

2019-12-15T19:04:04Z

This could cause a signal received during $(...) to corrupt memory.

45004: Fix typos in comments

2019-12-11T02:37:39Z

44841: Better checking of errors from "nice"

2019-10-16T16:41:41Z

44635: Don't apply STAT_NOPRINT to backgrounded jobs

2019-08-03T18:48:18Z

44480: Don't automatically close externally visible file descroptors.

2019-07-02T12:42:53Z

These are descriptors marked FDT_EXTERNAL. Make all sysopen'ed file descriptors FDT_EXTERNAL. Make =(...) call closem() consistent with other substitutions. Document file descriptors are left open.

44307: allow for atoi() returning a negative number

2019-05-19T22:14:01Z

44214: <(...) substitutions shouldn't grab the terminal

2019-04-10T08:44:58Z

44132: don't hash commands beginning with /

2019-03-19T14:09:50Z

43723: file descriptor could leak on fork error

2018-11-09T11:02:24Z

43660: extend 43653 when final exit is implicit.

2018-10-09T13:38:26Z

Combine logic for case after committed to exit (shell_exiting) with case where exit occurred in a function we nee to unwind (exit_pending). Add sarky note for future generations to be confused at.

43489: Add error checking on a new write() call.

2018-09-25T19:35:15Z

Silences a compiler warning (write(2) is declared with warn_unused_result in current Debian Unstable).

43464: Another attachtty() fix.

2018-09-16T18:13:38Z

If list_pipe_job triggered more than once we need to know the most recent process group leader, so record that both if the attach happened in the main shell on in entersubsh(). Also don't pass back proocess group for ESUB_ASYNC subshells.

43446: More entersubsh() / addproc() wiring.

2018-09-12T08:22:10Z

Fix additional races by passing back use of list_pipe_job from subshell.

43409: Fix process group setting in main shell.

2018-09-07T16:49:35Z

A newly forked subshell now reports back the process group it will be using for the main shell to record. This prevents an error where the shell incorrectly thought an exiting process owned the terminal and so grabbed it back, putting the foreground process into the background.

CVE-2018-0502, CVE-2018-13259: Fix two security issues in shebang line parsing.

2018-09-03T15:07:47Z

See NEWS for more information. Patch by Anthony Sottile and Buck Evan.

43156, 43157: Need to allow for extra space in gethere().

2018-07-10T13:42:02Z

The point can now increment twice per iteration.

users/23531: Error exit/return fixes.

2018-07-09T10:07:53Z

Fix regression with trap on left hand side of pipe. Fix forced return from shell structure within nested function. Fix tests exiting too early. Add new test case.

users/23472: Add $sysparams[procsubstpid] to zsh/system

2018-06-15T14:02:36Z

42101 (tweaked): assigning shell status to array was broken

2018-05-11T08:29:14Z

unposted (branch fork_early): Fork early for pipelines.

2018-05-01T09:27:35Z

If executing a command not at the end of a pipeline, and not optimised in the calling code, we can fork before "prefork" substitutions, at the same point as for background commands. Move fork before our preliminary scan of arguments to find a command: in the cases of early fork we don't need this information before forking. Ensure we _exit if forked in execcmd_exec(). Rationalise use of forks and pipes. Ensure we _exit instead of returning from execcmd_exec() if we have forked. Before the optimisation code after the fork always ran to the check at the end, but that code is overkill for the logic between the early fork and the existing one. Remove old workaround to fork in caller of execcmd for current shell constructs as no longer needed with early fork below. Close input of newly created pipe on fork (destined for RHS of pipe which we never execute): this replaces a workaround from zsh-workers/32171, commit 9887fc3d7b. Set last1 on early fork as needed by some instances of shell constructs on LHS of pipeline to know they are exiting.

47201: fix 42355 for multiple backslashes

2018-04-29T19:05:39Z

unposted: Correct process substitution buffer size in the PATH_DEV_FD codepath.

2018-04-25T15:31:39Z

42708: fix for process substitution.

2018-04-24T10:48:11Z

Don't close associated file descriptors in the closem() tidy up function as they should remain visible to external processes. Override if about to exit. Unit test for the failing case: note this relies on the existence of /proc/self/fd or equivalent.

42684 (with extra comments): Fork early if in bg.

2018-04-20T09:36:39Z

In execcmd the case of running the last command in a pipeline asynchronously for the purpose of & and &! is easy to work out, and we can avoid side effects and unnecessary execution time in the parent shell by forking earlier.

42630: Improve process group handling in pipelines.

2018-04-17T16:14:20Z

If process group leader exits, allow a newly forked process to become process leader. If a foreground job, reattach the shell to the terminal until that happens. Unblock signals when reading output for command subsitution so that we can do this reattaching immediately.

42624 (plus test): avoid freeing memory that's still needed

2018-04-12T21:15:29Z

This was occurring in a multiple function definition where a function name is duplicated.

42518, CVE-2018-1071: check bounds when copying path in hashcmd()

2018-03-24T14:04:02Z

42469: necessary repairs to 42465 found by "make check"

2018-03-18T23:49:42Z

42465: Pass up error status from readoutput().

2018-03-15T09:21:47Z

This improves the consistency of error reporting from $(...) constructs.

42355: Fix use of backslashes on here doc input.

2018-02-12T10:06:45Z

Handling of white space in particular was confusing and inconsistent with other shells.

42043: ZSH_DEBUG_CMD should not WARN_CREATE_GLOBAL

2018-01-14T11:07:32Z

33395: Improvments for function managment.

2017-12-22T19:51:04Z

Functions defined inside other fucntions needs file line number adding. Particularly useful for anonymous fucntions. Add flag to indicate a function is anonymous. Done up to now by comparing the name to a pointer but this is more consistent.

users/13148 (and workers/42060): Abort the command line when replying

2017-12-20T04:13:01Z

"[n] No" to the RM_STAR_SILENT prompt. Patch by Stephane, rebased by Mikael.

42123 (tweaked): take account of Dash in function names.

2017-12-15T09:00:37Z

Needed when comparing word code function name with autoload request. Add test.

41802 (minor tweaks): use heap during shell function call.

2017-10-04T08:18:51Z

Replaces stack for more efficient memory management. Also fix debug message when FUNCNEST is increased.

41787 (plus minor tweaks): use $FUNCSTACK for function nesting depth.

2017-10-02T08:44:26Z

Initialised from existing configuration value.

41789: Don't save fd if -1.

2017-09-29T15:29:49Z

We try to move an fd which isn't opend but it will feel. This needs handling specially in the new code for marking saved fd's.

41767: preserve nonzero exit status on interrupt

2017-09-28T03:34:11Z

41761: Ensure status from interrupt is propagated by builtin

2017-09-26T08:26:57Z

Updates for ksh array element syntax.

2017-09-24T16:33:07Z

Move detection of key/value pairs down into prefork(). Detect normal array assignment and [key]=val array assignemnt separately. Mark key / value pairs with Marker and pass up flag. Deal with marked triads specially later on.

41736: NO_INTERACTIVE_COMMENTS in $(...)

2017-09-20T17:17:59Z

First go at var=([key]=value) syntax.

2017-09-13T19:54:00Z

Works for both normal and typeset case, also var+=... Still to do: allow to be mixed with straight array assignment, improve typeset -p, implement [key]+=value.

41662: exec -a arguments weren't sanitised

2017-09-10T17:02:52Z

41608 (plus tests): restore ERR_EXIT before function.

2017-08-29T18:52:18Z

There was an exception to the usual ERR_EXIT pattern that causes problems when executing a function in an else branch. It seems the exception is no longer needed as the regression tests pass without it.

41590 modified as 41595: make ERR_RETURN more logical.

2017-08-24T18:46:20Z

It now operates separately at each function depth. To keep ERR_EXIT global, make the noerrexit variable usd bit flags. Extend tests.

posted but has not shown up: fix fd problem in subshell.

2017-08-15T19:49:23Z

Record fd's that have been saved in fdtable and if the shell forks close them as they will never be restored.

41464, 41466: With POSIX_BUILTINS, exec only executes sexternal commands.

2017-07-27T08:22:04Z

Document the default behaviour and add a test.

41317: use heap allocation in getoutputfile() to plug memory leak

2017-06-19T23:06:38Z

Heap is usable ever since addfilelist() of workers/31528 (commit 3c573222).

41043: Close pipes in shell when disowning background job.

2017-05-03T10:30:15Z

This was missing in the case of &,, causing a file descriptor leak.

41020: Fix "command -p"; "-p" was incorrectly left in command arguments

2017-04-28T09:22:43Z

Also add tests for known precommand modifier issues.

41012: Fix premature exit from nested function in EXIT trap.

2017-04-27T17:56:18Z

Also add check so we don't delay an exit if we were already in an EXIT trap for the main shell, as we should in that case leave immediately.

41008: Handle expansions for precommand modifiers

2017-04-26T08:44:38Z

40933: Ensure we get job text in sourced files.

2017-04-04T17:54:10Z

This was turned off explicitly as it was assumed not to be needed, but it is when code in a sourced file is suspended. We'll need to see if this makes start up inefficient.

40796: MAGIC_EQUAL_SUBST not needed with parsed assignment.

2017-03-09T09:52:23Z

If typeset family builtins are recognised as keywords then the value is handled as a separate expansion and we don't need the old magic behaviour, even if the option is set.

40760: Always tokenize unquoted - to Dash.

2017-03-07T10:43:58Z

This fixes use of pattern match character ranges in unusual contexts. Attempt to detect a tokenized - in cases where we don't care.

40654: exit cleanly from special POSIXBUILTINS in subshells

2017-03-04T23:25:17Z

This patch overloads the zsh_subshell counter which is meant for $ZSH_SUBSHELL, but no one has suggested a working alternate test.

40576 (tweaked): entersubsh(): unblock any signals that were blocked for trap handling

2017-02-20T21:26:49Z

Also small improvement to loop that resets trap handlers

40391: Add WARN_NESTED_VAR option and functions -W.

2017-01-23T09:50:57Z

These are companions to WARN_CREATED_GLOBAL, warning when a variable from an enclosing scope is altered.

40353 with tweaks to whence -v: extend directory cache use.

2017-01-16T10:31:56Z

Now used for all autoloaded functions after load, including those where the file was found along fpath, reducing duplication of directory names.

40342: Add directory name cache for autoload file paths.

2017-01-12T20:56:20Z

This renders "autoload /blah/blah/*" as efficient as use of fpath.

40335: More care with autoload function path.

2017-01-12T13:54:29Z

If doing "autoload -X", the path present might actually be location of file containing the function with the autoload -X. Add an explicit flag to say it's a directory for autoload.

unposted: more care with shell function filename.

2017-01-11T12:11:22Z

When updating shfunc structure to change filename we should be careful to free whatever's there already, we may now be the directory in which to find the function definition. After loading the field contains the full name of the file.

Add features associated with autoloading a function using an absolute

2017-01-11T11:26:13Z

path. -d defaults to normal fpath -r remembers the path without actually loading. May be combined with -d. -R does the same but it's an error if not found -X can now take a directory path: this is used to output not yet loaded functions that have an associated path.

Additional case for ERR_RETURN and ERR_EXIT.

2016-12-05T12:29:23Z

Don't trigger just because status is non-zero at end of current shell group.

Extra case for ERR_RETURN and ERR_EXIT.

2016-12-05T11:21:01Z

Don't trigger just because status is non-zero at end of complex shell construct as this may be a case we've already suppressed.

39958: Add extra byte to PATH_MAX allocations.

2016-11-17T19:49:17Z

This ensures we've got enough space for a null, although this isn't always needed.

39915: whence: Honor PATH_DIRS option for arguments that start with './' or '../'.

2016-11-17T15:28:42Z

While here, add some docstrings.

39901: No EXIT trap on LHS of pipeline.

2016-11-10T10:37:27Z

There is a special case if the LHS is a shell construct. Add unit tests for both cases.

39571: Fix ERR_EXIT bug with && and function.

2016-10-05T16:46:42Z

"foo && bar" inside a function could cause the code outside the function not to perform ERR_EXIT or ERR_RETURN when needed.

39568: "! " suppresses ERR_EXIT

2016-10-05T12:52:31Z

39566: Improve usefulness of command_not_found_handler.

2016-10-05T11:14:43Z

Don't behave as if command not found if return status is non-zero as this may simply be the return status of the replacement command. Let the function report a command not found instead.

39545: Add some missing unqueue_signals().

2016-10-03T12:43:20Z

All of these are added simply to fit existing logic in other branches.

39521: Refactor start of execcmd().

2016-10-03T08:59:01Z

By splitting into _analyse and _exec execpline2() has easier access to the state at the start of execution. Use this to ensure we fork if this is a builtin with no arguments.

zsh/Src/exec.c, branch zsh-5.9

50136: Fix =(nosuchcommand) race/deadlock first reported in workers/42609

50162: Fix multios with current-shell "exec" (aka nullexec).

50049: care with signed characters

49906 (Bart), 49911: Fixes to querying jobs in subshell.