feat: support CR (\r) line endings in addition to LF and CRLF#302617
feat: support CR (\r) line endings in addition to LF and CRLF#302617jaykae wants to merge 3 commits intomicrosoft:mainfrom
Conversation
📬 CODENOTIFYThe following users are being notified based on files changed in this PR: @bpaseroMatched files:
|
|
@microsoft-github-policy-service agree company="Alara Imaging" |
Add CR (carriage return, \r) as a first-class end-of-line sequence, addressing microsoft#35797. This enables proper handling of files using CR-only line endings, such as HL7v2 healthcare files, pre-OSX Mac files, and various industrial protocol formats. Changes across all layers: Core enums: - Add CR to EndOfLineSequence, EndOfLinePreference, DefaultEndOfLine - Rename StringEOL.Invalid to StringEOL.CR (same value, 3) - Update standaloneEnums.ts and monaco.d.ts Text buffer: - Widen type signatures to accept '\r' alongside '\r\n' and '\n' - Add CR case to _getEndOfLine() switch - Update EOL detection to recognize CR-only files - Update normalization condition for CR Text model & services: - Update setEOL/getEndOfLineSequence/pushEOL for 3-way logic - Update modelService config mapping for CR Configuration: - Add '\r' (CR) to files.eol setting enum Status bar & UI: - Add 'CR' label to status bar indicator - Add CR option to Change End of Line Sequence quick-pick Extension API: - Add EndOfLine.CR = 3 to public API (vscode.d.ts) - Update type converters and extHostDocumentData Tests: - Add eolCounter.test.ts for CR detection - Add CR tests to pieceTreeTextBuffer.test.ts - Add CR tests to textModel.test.ts Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
6b6a12e to
7188184
Compare
There was a problem hiding this comment.
Pull request overview
Adds first-class support for CR-only (\r) line endings across the editor stack to avoid corrupting CR-terminated files on open/edit/save, addressing #35797.
Changes:
- Extend core EOL enums/types and propagate
'\r'support through text buffer + text model logic. - Surface CR in configuration (
files.eol) and UI (status bar indicator + “Change End of Line Sequence” picker). - Expose CR through the extension API and add/extend tests for CR detection and behavior.
Reviewed changes
Copilot reviewed 17 out of 19 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| src/vs/editor/common/model.ts | Adds CR to core EOL enums and widens ITextBuffer.setEOL to accept '\r'. |
| src/vs/editor/common/core/misc/eolCounter.ts | Renames “Invalid” to CR in low-level EOL detection. |
| src/vs/editor/common/standalone/standaloneEnums.ts | Mirrors new CR enum values for standalone editor consumers. |
| src/vs/monaco.d.ts | Mirrors new CR enum values for Monaco typings. |
| src/vs/editor/common/model/pieceTreeTextBuffer/pieceTreeBase.ts | Widens internal EOL types to include '\r'. |
| src/vs/editor/common/model/pieceTreeTextBuffer/pieceTreeTextBuffer.ts | Adds CR preference handling and supports CR buffer EOL. |
| src/vs/editor/common/model/pieceTreeTextBuffer/pieceTreeTextBufferBuilder.ts | Detects CR-only files and supports CR defaults. |
| src/vs/editor/common/model/textModel.ts | Updates model EOL conversions for LF/CRLF/CR. |
| src/vs/editor/common/services/modelService.ts | Maps files.eol to CR and keeps models in sync with CR. |
| src/vs/editor/common/core/edits/stringEdit.ts | Normalizes \r during edit normalization and widens types. |
| src/vs/workbench/contrib/files/browser/files.contribution.ts | Adds '\r' to the files.eol setting enum + label. |
| src/vs/workbench/browser/parts/editor/editorStatus.ts | Adds CR label and quick-pick option for changing EOL. |
| src/vs/workbench/api/common/extHostTypes/textEdit.ts | Adds EndOfLine.CR = 3 to ext host types. |
| src/vs/workbench/api/common/extHostTypeConverters.ts | Converts CR between extension API and internal model enums. |
| src/vs/workbench/api/common/extHostDocumentData.ts | Exposes CR via document.eol mapping. |
| src/vscode-dts/vscode.d.ts | Adds EndOfLine.CR = 3 to the public extension API typings. |
| src/vs/editor/test/common/core/misc/eolCounter.test.ts | New test coverage for CR detection and mixed EOL cases. |
| src/vs/editor/test/common/model/pieceTreeTextBuffer/pieceTreeTextBuffer.test.ts | Adds CR buffer behavior tests (detection, setEOL, round-trip). |
| src/vs/editor/test/common/model/textModel.test.ts | Adds CR-focused model tests for value length and EOL changes. |
Comments suppressed due to low confidence (1)
src/vs/editor/common/model/pieceTreeTextBuffer/pieceTreeTextBuffer.ts:288
- With the current
StringEOLvalues,expectedStrEOLcan match mixed-EOL insert text when the buffer is in CR mode. For example, an inserted string containing both\nand\r\nwill producestrEOL === 3, which equalsStringEOL.CRin this PR, so the code will skip normalization and allow mixed EOLs into a CR buffer. FixingStringEOLto use non-overlapping bit values (or adjusting the comparison to properly detect single-EOL text) will prevent this.
const bufferEOL = this.getEOL();
const expectedStrEOL = (bufferEOL === '\r\n' ? StringEOL.CRLF : bufferEOL === '\r' ? StringEOL.CR : StringEOL.LF);
if (strEOL === StringEOL.Unknown || strEOL === expectedStrEOL) {
validText = op.text;
} else {
validText = op.text.replace(/\r\n|\r|\n/g, bufferEOL);
}
You can also share your feedback on Copilot code review. Take the survey.
- Change StringEOL.CR from 3 to 4 to avoid collision with LF|CRLF (1|2=3).
With CR=3, mixed LF+CRLF text was indistinguishable from CR-only text,
causing edit normalization to be silently skipped for CR buffers.
- Restore missing test('guess indentation 1') wrapper that was accidentally
removed when CR tests were inserted, leaving assertGuess() calls orphaned.
- Update eolCounter mixed-EOL test assertions for the new bitmask values
and add a mixed LF+CRLF test to verify CR=4 is distinct from LF|CRLF=3.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Adds first-class support for CR (\r) line endings across VS Code’s editor stack (buffer → model → workbench UI → extension API), addressing #35797 and preventing unwanted normalization/corruption of CR-only files.
Changes:
- Extend core EOL enums and EOL detection to represent CR alongside LF/CRLF.
- Propagate CR support through PieceTree text buffers, TextModel, model service/config mapping, and UI (status bar + quick pick).
- Expose CR in the public extension API and add/extend tests for detection and round-tripping.
Reviewed changes
Copilot reviewed 17 out of 19 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| src/vs/editor/common/model.ts | Adds CR to internal EOL enums and widens ITextBuffer EOL typing. |
| src/vs/editor/common/core/misc/eolCounter.ts | Renames/repurposes the “invalid” EOL flag to CR with a distinct bit value. |
| src/vs/editor/common/standalone/standaloneEnums.ts | Mirrors updated EOL enums for the standalone Monaco surface (generated output). |
| src/vs/monaco.d.ts | Mirrors updated EOL enums for Monaco’s declaration surface. |
| src/vs/editor/common/model/pieceTreeTextBuffer/pieceTreeBase.ts | Allows CR as a stored/normalized EOL in the core piece tree buffer. |
| src/vs/editor/common/model/pieceTreeTextBuffer/pieceTreeTextBuffer.ts | Supports CR for EOL preference resolution and edit EOL normalization. |
| src/vs/editor/common/model/pieceTreeTextBuffer/pieceTreeTextBufferBuilder.ts | Detects CR-only files and updates normalization conditions. |
| src/vs/editor/common/model/textModel.ts | Updates set/get/push EOL logic to handle CR as a third option. |
| src/vs/editor/common/services/modelService.ts | Maps files.eol = \r into DefaultEndOfLine and reconciles EOL on reload. |
| src/vs/editor/common/core/edits/stringEdit.ts | Extends EOL normalization utilities to include CR. |
| src/vs/workbench/contrib/files/browser/files.contribution.ts | Allows \r as a valid files.eol configuration value. |
| src/vs/workbench/browser/parts/editor/editorStatus.ts | Displays “CR” in the status bar and adds a CR pick option. |
| src/vs/workbench/api/common/extHostTypes/textEdit.ts | Adds EndOfLine.CR to the extension host type. |
| src/vs/workbench/api/common/extHostTypeConverters.ts | Converts CR between API and internal model enums. |
| src/vs/workbench/api/common/extHostDocumentData.ts | Exposes CR via TextDocument.eol to extensions. |
| src/vscode-dts/vscode.d.ts | Exposes CR in the public vscode.EndOfLine API. |
| src/vs/editor/test/common/core/misc/eolCounter.test.ts | Adds tests for CR detection and mixed-EOL bit flag behavior. |
| src/vs/editor/test/common/model/pieceTreeTextBuffer/pieceTreeTextBuffer.test.ts | Adds buffer-level CR detection/normalization and round-trip tests. |
| src/vs/editor/test/common/model/textModel.test.ts | Adds TextModel-level tests for CR preference, setEOL, and pushEOL behavior. |
You can also share your feedback on Copilot code review. Take the survey.
Add CR (carriage return, \r) as a first-class end-of-line sequence, addressing #35797. This enables proper handling of files using CR-only line endings, such as HL7v2 healthcare files, pre-OSX Mac files, and various industrial protocol formats.
FULL DISCLOSURE - Completely AI Generated used Copilot and Opus but human reviewed and tests passing before PR submission
Changes across all layers:
Core enums:
Text buffer:
Text model & services:
Configuration:
Status bar & UI:
Extension API:
Tests:
Walkthrough: Adding CR (
\r) Line Ending Support to VS Code (Updated 3/26/17 10PM EST)As someone not familiar with the
vscodecodebase, I asked Copilot to generate some documentation on the changesIssue: microsoft/vscode#35797
Branch:
feature/cr-line-endingsBackground
VS Code has always supported two types of line endings:
\n) — Used by Linux, macOS (10.0+), and most modern Unix systems\r\n) — Used by WindowsHowever, a third line ending exists: CR (
\r) — Used by pre-OSX classic Mac OS, and critically, still in active use today by:When VS Code opens a file with CR-only line endings, it silently normalizes them to CRLF, corrupting the file. The
files.eolsetting rejects"\r"with:This walkthrough explains every change needed to add CR as a first-class citizen throughout the VS Code editor stack.
Architecture Overview
Before diving into the code, it helps to understand how VS Code's EOL handling is layered:
Data flows bottom-up when opening a file (detect → buffer → model → UI) and top-down when the user changes settings or picks a new EOL (UI → model → buffer → normalize).
Layer 1: Core Enum Definitions
Every component in VS Code references EOL through a set of TypeScript enums. These are the foundation — everything else depends on them.
1a. The Three Internal Enums
File:
src/vs/editor/common/model.tsVS Code uses three separate enums for different purposes:
Why three enums? They serve different roles:
EndOfLinePreferenceincludesTextDefined(use whatever the buffer has) — used by APIs likegetValueInRange()where you might want the text in a specific formatDefaultEndOfLinestarts at 1 (not 0) — used in configuration where 0 would be falsyEndOfLineSequencestarts at 0 — used as the canonical identifier for a file's actual line endings1b. The EOL Detection Enum
File:
src/vs/editor/common/core/misc/eolCounter.tsThis enum is used internally by the low-level scanner that counts line endings in raw text:
Key insight: The values are chosen as bit flags so they can be OR'd together. When scanning text, the function does
eol |= StringEOL.CRevery time it finds a lone\r. A file with both CR and LF endings would haveeol = 4 | 1 = 5(distinct bits set for each type). The old code already detected lone CR — it just called it "Invalid" with the value3. We renamed itCRand changed its value to4to give it a distinct bit that cannot collide with any OR-combination of LF (1) and CRLF (2), which sum to3when both are present.The
countEOLfunction that uses this enum already handled CR correctly:1c. The Standalone/Monaco Enums
Files:
src/vs/editor/common/standalone/standaloneEnums.ts,src/vs/monaco.d.tsThese are mirror copies of the same enums, published for the Monaco editor (the standalone version of VS Code's editor). They are auto-generated files that need the same
CRvalues added. The changes are identical tomodel.ts— just add the CR member to each enum.Layer 2: Text Buffer — Where Text Lives
The PieceTree is VS Code's core data structure for storing file content. It's a balanced tree of string "pieces" that supports efficient insertion, deletion, and line-based access. EOL handling is deeply embedded here.
2a. PieceTreeBase — The Core Storage
File:
src/vs/editor/common/model/pieceTreeTextBuffer/pieceTreeBase.tsThe base class stores the EOL string and its length. Every type signature that previously accepted
'\r\n' | '\n'now accepts'\r\n' | '\n' | '\r':Critical insight: The
normalizeEOLmethod uses the regex/\r\n|\r|\n/gto replace all line endings. This regex already handles CR! The order matters —\r\nis matched first (before the standalone\r), preventing CRLF from being split into CR + LF. No regex changes were needed.2b. PieceTreeTextBuffer — The Wrapper
File:
src/vs/editor/common/model/pieceTreeTextBuffer/pieceTreeTextBuffer.tsThis wraps
PieceTreeBaseand adds higher-level operations. Three changes:1. Type signatures widened to accept
'\r':2. The
_getEndOfLineswitch — This converts theEndOfLinePreferenceenum to an actual string. We add the CR case:3. The edit validation mapping — When text is inserted into a buffer, VS Code checks if the inserted text's line endings match the buffer's EOL. If not, it normalizes them. The mapping from buffer EOL string to
StringEOLenum was binary and needed a third case:2c. PieceTreeTextBufferBuilder — The Factory
File:
src/vs/editor/common/model/pieceTreeTextBuffer/pieceTreeTextBufferBuilder.tsThis is where VS Code detects which EOL a file uses when it's first opened. The builder accumulates counts of
\r,\n, and\r\nas it reads file chunks, then the factory's_getEOLmethod decides which EOL "wins".Before (binary decision):
After (three-way decision):
Why the conservative detection? CR-only files are detected only when the file has exclusively CR endings (no LF or CRLF mixed in). Mixed files with some CR are still handled by the existing CRLF/LF heuristic. This prevents false positives — a file with accidental lone CRs won't suddenly be treated as a CR file.
The
create()method's normalization condition also needed updating to handle the case where the detected EOL is CR but the file contains other EOL types:Layer 3: Text Model — The Editor's View of the Buffer
File:
src/vs/editor/common/model/textModel.tsThe
TextModelis the high-level representation of a document. It wraps the buffer and provides the API that the rest of VS Code uses. Three methods needed updating from binary to three-way logic.3a.
setEOL— Change a file's line endingsConverts from the enum to the actual string. Was a simple ternary, now needs three branches:
3b.
getEndOfLineSequence— Query a file's line endingsConverts from the buffer's string back to the enum. Was a ternary, now an if-chain:
3c.
pushEOL— Change line endings with undo supportThis method compares the current EOL to the requested one. The old code used an inline binary conversion; we now reuse the
getEndOfLineSequence()method:Layer 4: Model Service — Configuration Bridge
File:
src/vs/editor/common/services/modelService.tsThe
ModelServicereads thefiles.eolconfiguration setting and translates it into the internal enum. Three places needed updating:4a. Reading configuration
4b. Applying configuration to models
When the setting changes and a model has only one line (no existing EOL to preserve), VS Code applies the new default:
4c. Reconciling file content with model
When a file is reloaded from disk, the model service syncs the EOL:
Layer 5: The
files.eolSettingFile:
src/vs/workbench/contrib/files/browser/files.contribution.tsThis is where the actual user-facing setting is defined. The change is simple — add
'\r'to the enum and its description:Important: The
textResourcePropertiesService.getEOL()method insrc/vs/workbench/services/textresourceProperties/common/textResourcePropertiesService.tsalready passes through the raw setting value for any non-'auto'string. No change was needed there — once the schema accepts'\r', the service will pass it through to the model layer automatically.Users can now write this in their
settings.json:{ "[hl7]": { "files.eol": "\r" } }Layer 6: Status Bar & UI
File:
src/vs/workbench/browser/parts/editor/editorStatus.tsThe bottom status bar shows the current file's line endings ("LF" or "CRLF") and lets users click to change them.
6a. Add the localized label
6b. Update the status bar display
The display logic was a binary ternary. Now it's a three-way chain:
6c. Update the "Change End of Line Sequence" quick-pick
When users click the status bar item, they get a picker. We add CR as an option:
Layer 7: Extension API
Extensions interact with VS Code through a typed API defined in
vscode.d.ts. Three files needed changes to expose CR to extensions.7a. Public API type definition
File:
src/vscode-dts/vscode.d.ts7b. Internal type implementation
File:
src/vs/workbench/api/common/extHostTypes/textEdit.ts7c. Type converters (internal ↔ extension API)
File:
src/vs/workbench/api/common/extHostTypeConverters.tsThese functions translate between the internal
EndOfLineSequenceand the publicEndOfLine:7d. Extension host document data
File:
src/vs/workbench/api/common/extHostDocumentData.tsWhen an extension accesses
document.eol, this getter maps the raw EOL string to the public enum:Layer 8: String Edit Utilities
File:
src/vs/editor/common/core/edits/stringEdit.tsThe
StringEditandStringReplacementclasses havenormalizeEOLmethods used during formatting operations. Their type signatures needed widening, and the regex inStringReplacementneeded to include\rin its match pattern:The regex change from
/\r\n|\n/gto/\r\n|\r|\n/gis important — the old regex would have left lone\rcharacters untouched when normalizing to LF or CRLF.Layer 9: Tests
Three test files were added or modified, covering ~170 lines of new test code.
9a. eolCounter.test.ts (NEW FILE)
File:
src/vs/editor/test/common/core/misc/eolCounter.test.tsTests the low-level
countEOLfunction's ability to detect CR line endings:9b. pieceTreeTextBuffer.test.ts
File:
src/vs/editor/test/common/model/pieceTreeTextBuffer/pieceTreeTextBuffer.test.tsA new
'CR line endings'test suite with 5 tests:9c. textModel.test.ts
File:
src/vs/editor/test/common/model/textModel.test.tsThree new tests at the TextModel level:
What Didn't Need to Change (and Why)
Several files were reviewed but needed no modification:
textResourcePropertiesService.tsfiles.eolstring for non-'auto'valuestextModelSearch.tslfCounteris only needed for CRLF→LF offset compensation (CRLF is 2 chars, LF is 1). CR is also 1 char, so no compensation neededtextModel.tsline 1314 (trailing CR stripping)\r— CR mode shouldn't strip trailing\rsince that's the actual line endingpieceTreeBase.tsnormalizeEOLregexSummary of All Changed Files
src/vs/editor/common/model.tssrc/vs/editor/common/core/misc/eolCounter.tsInvalid→CR, change value from3to4(distinct bit)src/vs/editor/common/standalone/standaloneEnums.tssrc/vs/monaco.d.tssrc/vs/editor/common/model/pieceTreeTextBuffer/pieceTreeBase.tssrc/vs/editor/common/model/pieceTreeTextBuffer/pieceTreeTextBuffer.tssrc/vs/editor/common/model/pieceTreeTextBuffer/pieceTreeTextBufferBuilder.tssrc/vs/editor/common/model/textModel.tssrc/vs/editor/common/services/modelService.tssrc/vs/editor/common/core/edits/stringEdit.tssrc/vs/workbench/contrib/files/browser/files.contribution.ts'\r'to settingsrc/vs/workbench/browser/parts/editor/editorStatus.tssrc/vs/workbench/api/common/extHostTypes/textEdit.tsCR = 3src/vs/workbench/api/common/extHostTypeConverters.tssrc/vs/workbench/api/common/extHostDocumentData.tssrc/vscode-dts/vscode.d.tsCR = 3src/vs/editor/test/common/core/misc/eolCounter.test.tssrc/vs/editor/test/common/model/pieceTreeTextBuffer/pieceTreeTextBuffer.test.tssrc/vs/editor/test/common/model/textModel.test.tsTotal: 19 files, +281 insertions, -45 deletions