diff options
Diffstat (limited to 'mnv/runtime/doc/develop.txt')
| -rw-r--r-- | mnv/runtime/doc/develop.txt | 757 |
1 files changed, 757 insertions, 0 deletions
diff --git a/mnv/runtime/doc/develop.txt b/mnv/runtime/doc/develop.txt new file mode 100644 index 0000000000..bc3cdba4a7 --- /dev/null +++ b/mnv/runtime/doc/develop.txt @@ -0,0 +1,757 @@ +*develop.txt* For MNV version 10.0. Last change: 2026 Feb 14 + + + MNV REFERENCE MANUAL by Bram Moolenaar + + +Development of MNV. *development* + +This text is important for those who want to be involved in further developing +MNV. + +1. Design goals |design-goals| +2. Design decisions |design-decisions| +3. Assumptions |design-assumptions| +4. Coding style |coding-style| +5. Policy |design-policy| + +See the file README.txt in the "src" directory for an overview of the source +code. + +MNV is open source software. Everybody is encouraged to contribute to help +improving MNV. For sending patches a unified diff "diff -u" is preferred. +You can create a pull request on github, but it's not required. +Also see http://mnv.wikia.com/wiki/How_to_make_and_submit_a_patch. + +============================================================================== +1. Design goals *design-goals* + +Most important things come first (roughly). + +Note that quite a few items are contradicting. This is intentional. A +balance must be found between them. + + +MNV IS... VI COMPATIBLE *design-compatible* + +First of all, it should be possible to use MNV as a drop-in replacement for +Vi. When the user wants to, MNV can be used in compatible mode and hardly +any differences with the original Vi will be noticed. + +Exceptions: +- We don't reproduce obvious Vi bugs in MNV. +- There are different versions of Vi. I am using Version 3.7 (6/7/85) as a + reference. But support for other versions is also included when possible. + The Vi part of POSIX is not considered a definitive source. +- MNV adds new commands, you cannot rely on some command to fail because it + didn't exist in Vi. +- MNV will have a lot of features that Vi doesn't have. Going back from MNV + to Vi will be a problem, this cannot be avoided. +- Some things are hardly ever used (open mode, sending an e-mail when + crashing, etc.). Those will only be included when someone has a good reason + why it should be included and it's not too much work. +- For some items it is debatable whether Vi compatibility should be + maintained. There will be an option flag for these. + + +MNV IS... IMPROVED *design-improved* + +The IMproved bits of MNV should make it a better Vi, without becoming a +completely different editor. Extensions are done with a "Vi spirit". +- Use the keyboard as much as feasible. The mouse requires a third hand, + which we don't have. Many terminals don't have a mouse. +- When the mouse is used anyway, avoid the need to switch back to the + keyboard. Avoid mixing mouse and keyboard handling. +- Add commands and options in a consistent way. Otherwise people will have a + hard time finding and remembering them. Keep in mind that more commands and + options will be added later. +- A feature that people do not know about is a useless feature. Don't add + obscure features, or at least add hints in documentation that they exist. +- Minimize using CTRL and other modifiers, they are more difficult to type. +- There are many first-time and inexperienced MNV users. Make it easy for + them to start using MNV and learn more over time. +- There is no limit to the features that can be added. Selecting new features + is one based on (1) what users ask for, (2) how much effort it takes to + implement and (3) someone actually implementing it. + + +MNV IS... MULTI PLATFORM *design-multi-platform* + +MNV tries to help as many users on as many platforms as possible. +- Support many kinds of terminals. The minimal demands are cursor positioning + and clear-screen. Commands should only use key strokes that most keyboards + have. Support all the keys on the keyboard for mapping. +- Support many platforms. A condition is that there is someone willing to do + MNV development on that platform, and it doesn't mean messing up the code. +- Support many compilers and libraries. Not everybody is able or allowed to + install another compiler or GUI library. +- People switch from one platform to another, and from GUI to terminal + version. Features should be present in all versions, or at least in as many + as possible with a reasonable effort. Try to avoid that users must switch + between platforms to accomplish their work efficiently. +- That a feature is not possible on some platforms, or only possible on one + platform, does not mean it cannot be implemented. [This intentionally + contradicts the previous item, these two must be balanced.] + + +MNV IS... WELL DOCUMENTED *design-documented* + +- A feature that isn't documented is a useless feature. A patch for a new + feature must include the documentation. +- Documentation should be comprehensive and understandable. Using examples is + recommended. +- Don't make the text unnecessarily long. Less documentation means that an + item is easier to find. + + +MNV IS... HIGH SPEED AND SMALL IN SIZE *design-speed-size* + +Using MNV must not be a big attack on system resources. Keep it small and +fast. +- Computers are becoming faster and bigger each year. MNV can grow too, but + no faster than computers are growing. Keep MNV usable on older systems. +- Many users start MNV from a shell very often. Startup time must be short. +- Commands must work efficiently. The time they consume must be as small as + possible. Useful commands may take longer. +- Don't forget that some people use MNV over a slow connection. Minimize the + communication overhead. +- Items that add considerably to the size and are not used by many people + should be a feature that can be disabled. +- MNV is a component among other components. Don't turn it into a massive + application, but have it work well together with other programs. + + +MNV IS... MAINTAINABLE *design-maintain* + +- The source code should not become a mess. It should be reliable code. +- Use the same layout in all files to make it easy to read |coding-style|. +- Use comments in a useful way! Quoting the function name and argument names + is NOT useful. Do explain what they are for. +- Porting to another platform should be made easy, without having to change + too much platform-independent code. +- Use the object-oriented spirit: Put data and code together. Minimize the + knowledge spread to other parts of the code. + + +MNV IS... FLEXIBLE *design-flexible* + +MNV should make it easy for users to work in their preferred styles rather +than coercing its users into particular patterns of work. This can be for +items with a large impact (e.g., the 'compatible' option) or for details. The +defaults are carefully chosen such that most users will enjoy using MNV as it +is. Commands and options can be used to adjust MNV to the desire of the user +and its environment. + + +MNV IS... NOT *design-not* + +- MNV is not a shell or an Operating System. It does provide a terminal + window, in which you can run a shell or debugger. E.g. to be able to do + this over an ssh connection. But if you don't need a text editor with that + it is out of scope (use something like screen or tmux instead). + A satirical way to say this: "Unlike Emacs, MNV does not attempt to include + everything but the kitchen sink, but some people say that you can clean one + with it. ;-)" + To use MNV with gdb see |terminal-debugger|. Other (older) tools can be + found at http://clewn.sf.net. +- MNV is not a fancy GUI editor that tries to look nice at the cost of + being less consistent over all platforms. But functional GUI features are + welcomed. + + +============================================================================== +2. Design decisions *design-decisions* + +Folding + +Several forms of folding should be possible for the same buffer. For example, +have one window that shows the text with function bodies folded, another +window that shows a function body. + +Folding is a way to display the text. It should not change the text itself. +Therefore the folding has been implemented as a filter between the text stored +in a buffer (buffer lines) and the text displayed in a window (logical lines). + + +Naming the window + +The word "window" is commonly used for several things: A window on the screen, +the xterm window, a window inside MNV to view a buffer. +To avoid confusion, other items that are sometimes called window have been +given another name. Here is an overview of the related items: + +screen The whole display. For the GUI it's something like 1024x768 + pixels. The MNV shell can use the whole screen or part of it. +shell The MNV application. This can cover the whole screen (e.g., + when running in a console) or part of it (xterm or GUI). +window View on a buffer. There can be several windows in MNV, + together with the command line, menubar, toolbar, etc. they + fit in the shell. + + +Spell checking *develop-spell* + +When spell checking was going to be added to MNV a survey was done over the +available spell checking libraries and programs. Unfortunately, the result +was that none of them provided sufficient capabilities to be used as the spell +checking engine in MNV, for various reasons: + +- Missing support for multibyte encodings. At least UTF-8 must be supported, + so that more than one language can be used in the same file. + Doing on-the-fly conversion is not always possible (would require iconv + support). +- For the programs and libraries: Using them as-is would require installing + them separately from MNV. That's mostly not impossible, but a drawback. +- Performance: A few tests showed that it's possible to check spelling on the + fly (while redrawing), just like syntax highlighting. But the mechanisms + used by other code are much slower. Myspell uses a hashtable, for example. + The affix compression that most spell checkers use makes it slower too. +- For using an external program like aspell a communication mechanism would + have to be setup. That's complicated to do in a portable way (Unix-only + would be relatively simple, but that's not good enough). And performance + will become a problem (lots of process switching involved). +- Missing support for words with non-word characters, such as "Etten-Leur" and + "et al.", would require marking the pieces of them OK, lowering the + reliability. +- Missing support for regions or dialects. Makes it difficult to accept + all English words and highlight non-Canadian words differently. +- Missing support for rare words. Many words are correct but hardly ever used + and could be a misspelled often-used word. +- For making suggestions the speed is less important and requiring to install + another program or library would be acceptable. But the word lists probably + differ, the suggestions may be wrong words. + + +Spelling suggestions *develop-spell-suggestions* + +For making suggestions there are two basic mechanisms: +1. Try changing the bad word a little bit and check for a match with a good + word. Or go through the list of good words, change them a little bit and + check for a match with the bad word. The changes are deleting a character, + inserting a character, swapping two characters, etc. +2. Perform soundfolding on both the bad word and the good words and then find + matches, possibly with a few changes like with the first mechanism. + +The first is good for finding typing mistakes. After experimenting with +hashtables and looking at solutions from other spell checkers the conclusion +was that a trie (a kind of tree structure) is ideal for this. Both for +reducing memory use and being able to try sensible changes. For example, when +inserting a character only characters that lead to good words need to be +tried. Other mechanisms (with hashtables) need to try all possible letters at +every position in the word. Also, a hashtable has the requirement that word +boundaries are identified separately, while a trie does not require this. +That makes the mechanism a lot simpler. + +Soundfolding is useful when someone knows how the words sounds but doesn't +know how it is spelled. For example, the word "dictionary" might be written +as "daktonerie". The number of changes that the first method would need to +try is very big, it's hard to find the good word that way. After soundfolding +the words become "tktnr" and "tkxnry", these differ by only two letters. + +To find words by their soundfolded equivalent (soundalike word) we need a list +of all soundfolded words. A few experiments have been done to find out what +the best method is. Alternatives: +1. Do the sound folding on the fly when looking for suggestions. This means + walking through the trie of good words, soundfolding each word and + checking how different it is from the bad word. This is very efficient for + memory use, but takes a long time. On a fast PC it takes a couple of + seconds for English, which can be acceptable for interactive use. But for + some languages it takes more than ten seconds (e.g., German, Catalan), + which is unacceptably slow. For batch processing (automatic corrections) + it's too slow for all languages. +2. Use a trie for the soundfolded words, so that searching can be done just + like how it works without soundfolding. This requires remembering a list + of good words for each soundfolded word. This makes finding matches very + fast but requires quite a lot of memory, in the order of 1 to 10 Mbyte. + For some languages more than the original word list. +3. Like the second alternative, but reduce the amount of memory by using affix + compression and store only the soundfolded basic word. This is what Aspell + does. Disadvantage is that affixes need to be stripped from the bad word + before soundfolding it, which means that mistakes at the start and/or end + of the word will cause the mechanism to fail. Also, this becomes slow when + the bad word is quite different from the good word. + +The choice made is to use the second mechanism and use a separate file. This +way a user with sufficient memory can get very good suggestions while a user +who is short of memory or just wants the spell checking and no suggestions +doesn't use so much memory. + + +Word frequency + +For sorting suggestions it helps to know which words are common. In theory we +could store a word frequency with the word in the dictionary. However, this +requires storing a count per word. That degrades word tree compression a lot. +And maintaining the word frequency for all languages will be a heavy task. +Also, it would be nice to prefer words that are already in the text. This way +the words that appear in the specific text are preferred for suggestions. + +What has been implemented is to count words that have been seen during +displaying. A hashtable is used to quickly find the word count. The count is +initialized from words listed in COMMON items in the affix file, so that it +also works when starting a new file. + +This isn't ideal, because the longer MNV is running the higher the counts +become. But in practice it is a noticeable improvement over not using the +word count. + +============================================================================== +3. Assumptions *design-assumptions* + +The following sections define the portability and compatibility constraints +that all MNV code and build tools must adhere to. + + +MAKEFILES *assumptions-makefiles* + *POSIX.1-2001* + +MNV's main Makefiles target maximum portability, relying solely on features +defined in POSIX.1-2001 `make` and ignoring later POSIX standards or GNU/BSD +extensions. In practical terms, avoid: + + - % pattern rules + - modern assignment (`:=`, `::=`) outside POSIX.1-2001 + - special targets (`.ONESHELL`, `.NOTPARALLEL`, `.SILENT`, ...) + - order-only prerequisites (`|`) or automatic directory creation + - GNU/BSD conditionals (`ifdef`, `ifndef`, `.for`/`.endfor`, ...) + +Since POSIX.1-2001 supports only traditional suffix rules, every object built +in a separate directory must have an explicit rule. For example: + + objects/evalbuffer.o: evalbuffer.c + $(CCC) -o $@ evalbuffer.c + +This verbosity ensures that the same Makefile builds MNV unchanged with the +default `make` on Linux, *BSD, macOS, Solaris, AIX, HP-UX and virtually any +Unix-like OS. + +Some platform-specific Makefiles (e.g., for Windows, NSIS, or Cygwin) may use +more advanced features when compatibility with basic make is not required. + + +C COMPILER *assumptions-C-compiler* + *ANSI-C* *C89* *C90* *C95* *C99* + +MNV strives for maximum portability (see |design-multi-platform|) and must +still build with Compaq C V6.4-005 on OpenVMS VAX V7.3. + +Therefore, the latest ISO C standard we follow is: + + `C95` (ISO/IEC 9899:1990/AMD1:1995) + +In addition, the following `C99` features are explicitly allowed: + - `//` comments, as required by |style-comments|; + - Mixed declarations and statements in a block; + - Variadic macros `(..., __VA_ARGS__)`; + - Trailing comma in `enum` lists; + - `_Bool` type (for `bool`, `true` and `false`); + - `__func__` predefined identifier; + - `inline` functions (use `static inline` for portability); + - Compound literals `(type){ initializer-list }`; + - Logical source lines up to 4095 characters. + +Platform-specific code may use any newer compiler features supported on that +platform. + + +SIZE OF VARIABLES *assumptions-variables* + +We follow POSIX.1-2001 (SUSv3) for type sizes, which in practice means: + + char_u 8-bit unsigned + int 32-bit or larger signed + unsigned 32-bit or larger unsigned + + +FUNCTION PROTOTYPES *assumptions-prototypes* + +MNV does not use conventional header files (`.h`) for most internal function +prototypes. Instead, the current architecture uses individual `.pro` files in +the `src/proto/` directory, with one `.pro` file per `.c` file. + +Unlike traditional self-contained header files, these `.pro` files do not +contain API documentation, struct and enum definitions, or other declarations; +only function prototypes. + +The `make proto` target in `src/Makefile` automates updating most of the .pro +files using the Python script proto/gen_prototypes.py, which relies on the +python3-clang module. Note that a few proto files are hand edited. + +============================================================================== +4. Coding style *coding-style* + +These are the rules to use when making changes to the MNV source code. Please +stick to these rules, to keep the sources readable and maintainable. + +This list is not complete. Look in the source code for more examples. + +The code repository contains an editorconfig file, that can be used together +with the distributed editorconfig plugin |editorconfig-install| to ensure the +recommended style is followed. + + +MAKING CHANGES *style-changes* + +The basic steps to make changes to the code: +1. Get the code from github. That makes it easier to keep your changed + version in sync with the main code base (it may be a while before your + changes will be included). +2. Adjust the documentation. Doing this first gives you an impression of how + your changes affect the user. +3. Make the source code changes. +4. Check ../doc/todo.txt if the change affects any listed item. +5. Add a test to src/testdir to verify the new behaviour and ensure it won't + regress in the future. +6. Make a patch with "git diff". +7. Make a note about what changed, preferably mentioning the problem and the + solution. Send an email to the |mnv-dev| maillist with an explanation and + include the diff. + +For any non-trivial change, please always create a pull request on github, +since this triggers the test suite. + +A PR should ideally contain a single commit for a single logical change. +However, you can include several commits if you want to group multiple +logical, atomic changes in one PR. This can also make longer PRs easier to +review. Be sure to describe the reasoning for your changes in each commit +message, as this greatly helps with the review process. In cases where each +commit handles different logical changes, they will also be applied as +separate patches in MNV's repository. + + *style-clang-format* +sound.c and sign.c can be (semi-) automatically formatted using the +`clang-format` formatter according to the distributed .clang-format file. +Other source files do not yet correspond to the .clang-format file. This may +change in the future and they may be reformatted as well. + + +COMMENTS *style-comments* + +Try to avoid putting multiline comments inside a function body: if the +function is so complex that you need to separately comment parts of it, you +should probably rethink the structure of the function. + +For file headers and function descriptions use: > + /* + * Description + */ +< +For everything else use: > + // comment +< + +INDENTATION *style-indentation* + +We use 4 space to indent the code. If you are using MNV to edit the source, +you don't need to do anything due to the |modeline|. + +For other editors an `.editorconfig` is provided at the root of the repo. + +For the source files `sign.c` and `sound.c` and any new file use only spaces, +no tabs. In addition, any new file must include a modeline with `set et` to +pass the indentation test. + + +DECLARATIONS *style-declarations* + +Declare, when possible, `for` loop variables in the guard: +OK: > + for (int i = 0; i < len; ++i) +< +Wrong: > + int i; + for (i = 0; i < len; ++i) +< +Always declare a variable with a default value: +OK: > + int n = 0; + int *ptr = NULL; +< +Wrong: > + int n; + int *ptr; +< + +BRACES *style-braces* + +All curly braces must be returned onto a new line: +OK: > + if (cond) + { + cmd; + cmd; + } + else + { + cmd; + cmd; + } +< +Wrong: > + if (cond) { + cmd; + cmd; + } else { + cmd; + cmd; + } +< +OK: > + while (cond) + { + cmd; + cmd; + } +< +Wrong: > + while (cond) { + cmd; + cmd; + } +< +OK: > + do + { + cmd; + cmd; + } while (cond); +< +or > + do + { + cmd; + cmd; + } + while (cond); +< +Wrong: > + do { + cmd; + cmd; + } while (cond); +< + +TYPES *style-types* + +Use descriptive types. These are defined in src/mnv.h, src/structs.h etc. +Note that all custom types are postfixed with "_T" + +Example: > + linenr_T + buf_T + pos_T +< + +SPACES AND PUNCTUATION *style-spaces* + +No space between a function name and the bracket: + +OK: func(arg); +Wrong: func (arg); + +Do use a space after `if`, `while`, `switch`, etc. + +OK: if (arg) for (;;) +Wrong: if(arg) for(;;) + +Use a space after a comma or semicolon: + +OK: func(arg1, arg2); for (i = 0; i < 2; ++i) +Wrong: func(arg1,arg2); for (i = 0;i < 2;++i) + +Use a space before and after '=', '+', '/', etc. + +OK: var = a * 5; +Wrong: var=a*5; + +Use empty lines to group similar actions together. + +OK: > + msg_puts_title(_("\n--- Signs ---")); + msg_putchar('\n'); + + if (rbuf == NULL) + buf = firstbuf; + else + buf = rbuf; + + while (buf != NULL && !got_int) +< +Wrong: > + msg_puts_title(_("\n--- Signs ---")); + msg_putchar('\n'); + if (rbuf == NULL) + buf = firstbuf; + else + buf = rbuf; + while (buf != NULL && !got_int) +< + +FUNCTIONS *style-functions* + +Use function declarations with the return type on a separate indented line. + +OK: > + int + function_name(int arg1, int arg2) + { + } +< +Wrong: > + int function_name(int arg1, int arg2) + { + } +< + +Give meaningful names to function parameters. + + +USE OF COMMON FUNCTIONS *style-common-functions* + +Some functions that are common to use, have a special MNV version. Always +consider using the MNV version, because they were introduced with a reason. + +NORMAL NAME MNV NAME DIFFERENCE OF MNV VERSION +free() mnv_free() Checks for freeing NULL +malloc() alloc() Checks for out of memory situation +malloc() lalloc() Like alloc(), but has long argument +strcpy() STRCPY() Includes cast to (char *), for char_u * args +strchr() mnv_strchr() Accepts special characters +strrchr() mnv_strrchr() Accepts special characters +isspace() mnv_isspace() Can handle characters > 128 +iswhite() mnv_iswhite() Only TRUE for tab and space +memcpy() mch_memmove() Handles overlapped copies +bcopy() mch_memmove() Handles overlapped copies +memset() mnv_memset() Uniform for all systems + + +NAMES *style-names* + +Function names can not be more than 31 characters long (because of VMS). + +Don't use "delete" or "this" as a variable name, C++ doesn't like it. + +Because of the requirement that MNV runs on as many systems as possible, we +need to avoid using names that are already defined by the system. This is a +list of names that are known to cause trouble. The name is given as a regexp +pattern. + +is.*() POSIX, ctype.h +to.*() POSIX, ctype.h + +d_.* POSIX, dirent.h +l_.* POSIX, fcntl.h +gr_.* POSIX, grp.h +pw_.* POSIX, pwd.h +sa_.* POSIX, signal.h +mem.* POSIX, string.h +str.* POSIX, string.h +wcs.* POSIX, string.h +st_.* POSIX, stat.h +tms_.* POSIX, times.h +tm_.* POSIX, time.h +c_.* POSIX, termios.h +MAX.* POSIX, limits.h +__.* POSIX, system +_[A-Z].* POSIX, system +E[A-Z0-9]* POSIX, errno.h + +.*_t POSIX, for typedefs. Use .*_T instead. + +wait don't use as argument to a function, conflicts with types.h +index shadows global declaration +time shadows global declaration +new C++ reserved keyword + +clear Mac curses.h +echo Mac curses.h +instr Mac curses.h +meta Mac curses.h +newwin Mac curses.h +nl Mac curses.h +overwrite Mac curses.h +refresh Mac curses.h +scroll Mac curses.h +typeahead Mac curses.h + +basename() GNU string function +dirname() GNU string function +get_env_value() Linux system function + + +VARIOUS *style-various* + +Define'd names should be uppercase: > + #define SOME_THING +< + +Features always start with "FEAT_": > + #define FEAT_FOO +< + +Don't use '\"', some compilers can't handle it. '"' works fine. + +Don't use: > + #if HAVE_SOME +< +Some compilers can't handle that and complain that "HAVE_SOME" is not defined. +Use > + #ifdef HAVE_SOME +< +or > + #if defined(HAVE_SOME) +< + +STYLE *style-examples* + +One statement per line. + +Wrong: if (cond) a = 1; + +OK: if (cond) + a = 1; + +Wrong: while (cond); + +OK: while (cond) + ; + +Wrong: do a = 1; while (cond); + +OK: do + a = 1; + while (cond); + + +============================================================================== +5. Policy *design-policy* *new-features* *deprecated-features* + +The time between either a new minor (e.g. 10.0.0) or major (e.g. 10.0) version +is released is called a development cycle. Within the development cycle each +single change to the C core will receive a new increased human-readable patch +number in order to reference each specific patch release. A typical +development release cycle may last several years and accumulate about 1500 - +2500 patch numbers. + +Before a release is made, a stability period will be announced. During this +time, only clear bug fixes, security fixes, documentation changes, translation +updates and runtime file updates will be accepted (provided they do not +introduce backwards-incompatible changes), concentrating on polishing up the +upcoming release. + +New features are accepted only within a development cycle, but not within the +stability period. During the cycle, new features may be developed and are +allowed to change, but they must be settled before the cycle closes. + +Once a minor release has been made, features included in that release must not +receive any backwards-incompatible changes. Later patches are expected to +preserve compatibility for the C core of MNV. Runtime files are handled a bit +more flexibly to give runtime files maintainers a chance to change old +behaviour. + +Within a development cycle, features may be marked as deprecated. Deprecated +features can be disabled at compile time through an appropriate switch. After +a new release, deprecated features may be removed completely in a following +cycle. + + mnv:tw=78:ts=8:noet:ft=help:norl: |