Domain-specific language

Overview

A domain-specific language is created specifically to solve problems in a specific domain and is not intended to be able to solve problems outside of it (although that may be technically possible). In contrast, general-purpose languages are created to solve problems in many domains.

In design and implementation

Domain-specific languages are languages (or often, declared syntaxes or grammars) with very specific goals in design and implementation. A domain-specific language can be one of a visual diagramming language, such as those created by the Generic Eclipse Modeling System, programmatic abstractions, such as the Eclipse Modeling Framework, or textual languages. For instance, the command line utility grep has a regular expression syntax which matches patterns in lines of text. The sed utility defines a syntax for matching and replacing regular expressions. Often, these tiny languages can be used together inside a shell to perform more complex programming tasks.

The line between domain-specific languages and scripting languages is somewhat blurred, but domain-specific languages often lack low-level functions for file system access, interprocess control, and other functions that characterize full-featured programming languages, scripting or otherwise. Many domain-specific languages do not compile to bytecode or executable code, but to various kinds of media objects: Csound compiles to audio files, and a ray-tracing domain-specific language like Persistence of Vision Ray Tracer (POV-Ray) compiles to graphics files.

Data definition languages

A data definition language like SQL presents an interesting case: it can be deemed a domain-specific language because it is specific to a specific domain (in SQL's case, accessing and managing relational databases), and is often called from another application, but SQL has more keywords and functions than many scripting languages, and is often viewed as a full language, in part because of the prevalence of database manipulation in programming and the amount of mastery needed to be expert in the language.

Further blurring this line, many domain-specific languages have exposed APIs, and can be accessed from other programming languages without breaking the flow of execution or calling a separate process, and can thus operate as programming libraries.

Programming tools

Some domain-specific languages expand over time to include full-featured programming tools, which further complicates the question of whether a language is domain-specific or not. A good example is the functional language Extensible Stylesheet Language Transformation (XSLT), specifically designed to transform one XML graph into another, which has been extended since its inception to allow (specifically in its 2.0 version) various forms of file system interaction, string and date manipulation, and data typing.

In model-driven engineering, many examples of domain-specific languages may be found like Object Constraint Language (OCL), a language for decorating models with assertions or Query/View/Transformation (QVT), a domain-specific transformation language. However, languages like Unified Modeling Language (UML) are typically general-purpose modeling languages.

To summarize, an analogy is useful: a very little language is like a knife, which can be used in thousands of different ways, from cutting food to self-defense. A domain-specific language is like an electric drill: it is a powerful tool with a wide variety of uses, but a specific context, namely, making holes in things. A general purpose language is a complete workbench, with a variety of tools intended for performing a variety of tasks. Domain-specific languages should be used by programmers who, looking at their current workbench, realize they need a better drill and find that a given domain-specific language provides exactly that.

Domain-specific language topics

External and embedded domain specific languages

DSLs implemented via an independent interpreter or compiler are termed external domain specific languages. Well known examples include TeX or AWK. A separate category termed embedded (or internal) domain specific languages are typically implemented within a host language as a library and tend to be limited to the syntax of the host language, though this depends on host language abilities.

Usage patterns

There are several usage patterns for domain-specific languages:

• Processing with standalone tools, invoked via direct user operation, often on the command line or from a Makefile (e.g., grep for regular expression matching, sed, lex, yacc, the GraphViz toolset, etc.)
• Domain-specific languages which are implemented using programming language macro systems, and which are converted or expanded into a host general purpose language at compile-time or realtime
• An embedded domain-specific language (eDSL) also known as an internal domain-specific language, is a DSL that is implemented as a library in a "host" programming language. The embedded domain-specific language leverages the syntax, semantics and runtime system–environment (sequencing, conditionals, iteration, functions, etc.) and adds domain-specific primitives that allow programmers to use the "host" programming language to create programs that generate code in the "target" programming language. Multiple eDSLs can easily be combined into a single program and the facilities of the host language can be used to extend an existing eDSL. Other possible advantages using an eDSL are improved type safety and better integrated development environment (IDE) tooling. eDSL examples: SQLAlchemy "Core" an SQL eDSL in Python, JOOQ Object Oriented Querying (jOOQ) an SQL eDSL in Java, Language Integrated Query's (LINQ) "method syntax" an SQL eDSL in C#, and kotlinx.html a Kotlin-DSL for HTML.
• Domain-specific languages which are called (at runtime) from programs written in general purpose languages like C or Perl, to perform a specific function, often returning the results of operation to the "host" programming language for further processing; generally, an interpreter or virtual machine for the domain-specific language is embedded into the host application (e.g., format strings, a regular expression engine)
• Domain-specific languages which are embedded into user applications (e.g., macro languages within spreadsheets) and which are (1) used to execute code that is written by users of the application, (2) dynamically generated by the application, or (3) both.

Design goals

Adopting a domain-specific language approach to software engineering involves both risks and opportunities. The well-designed domain-specific language manages to find the proper balance between these.

Domain-specific languages have important design goals that contrast with those of general-purpose languages:

• Domain-specific languages are less comprehensive.
• Domain-specific languages are much more expressive in their domain.
• Domain-specific languages should exhibit minimal redundancy.

Idioms

In programming, idioms are methods imposed by programmers to handle common development tasks, e.g.:

• Ensure data is saved before the window is closed.
• Edit code whenever command-line parameters change because they affect program behavior.

General purpose programming languages rarely support such idioms as part of their syntax or runtime, but domain-specific languages sometimes do, e.g.:

• A script can automatically save data.
• A domain-specific language can parameterize command line input.

Examples

Examples of domain-specific languages include Verilog and VHDL hardware description languages, MATLAB and GNU Octave for matrix programming, Mathematica, Maple and Maxima for symbolic mathematics, Specification and Description Language for reactive and distributed systems, spreadsheet formulas and macros, SQL for relational database queries, Yacc grammars for creating parsers, regular expressions for pattern matching, the Generic Eclipse Modeling System for creating diagramming languages, Csound for sound and music synthesis, and the input languages of GraphViz and GrGen, software packages used for graph layout and graph rewriting, HashiCorp Configuration Language used for Terraform and other HashiCorp tools, Puppet also has its own configuration language.

GameMaker Language

GML is a domain-specific language used by GameMaker Studio designed to help novice programmers learn the fundamentals of coding more easily. It functions as a blend of several languages, including Delphi, C++, and BASIC. Most GML functions, once compiled, call runtime functions written in the specific language of the target platform; consequently, their final implementation remains hidden from the user. The language's primary goal is to lower the barrier to entry for game development. The GameMaker runtime, which manages the main game loop and handles function implementation, allows a simple game to use only a few lines of code instead of thousands.

ColdFusion Markup Language

ColdFusion's associated scripting language is another example of a domain-specific language for data-driven websites. The language is used to weave together languages and services such as Java, .NET, C++, SMS, email, email servers, http, ftp, exchange, directory services, and file systems for use in websites.

The ColdFusion Markup Language (CFML) includes a set of tags that can be used in ColdFusion pages to interact with data sources, manipulate data, and display output. CFML tag syntax is similar to HTML element syntax.

FilterMeister

FilterMeister is a programming environment, with a programming language that is based on C, for the specific purpose of creating Photoshop-compatible image processing filter plug-ins; FilterMeister runs as a Photoshop plug-in itself and it can load and execute scripts or compile and export them as independent plug-ins. Although the FilterMeister language reproduces a significant portion of the C language and function library, it contains only those features which can be used within the context of Photoshop plug-ins and adds a number of specific features only useful in this specific domain.

MediaWiki templates

The Template feature of MediaWiki is an embedded domain-specific language whose fundamental purpose is to support the creation of page templates and the transclusion (inclusion by reference) of MediaWiki pages into other MediaWiki pages.

Software engineering uses

There has been much interest in domain-specific languages to improve the productivity and quality of software engineering. Domain-specific language could possibly provide a robust set of tools for efficient software engineering. Such tools are beginning to make their way into the development of critical software systems.

The Software Cost Reduction Toolkit is an example of this. The toolkit is a suite of utilities including a specification editor to create a requirements specification, a dependency graph browser to display variable dependencies, a consistency checker to catch missing cases in well-formed formulas in the specification, a model checker and a theorem prover to check program properties against the specification, and an invariant generator that automatically constructs invariants based on the requirements.

A newer development is language-oriented programming, an integrated software engineering methodology based mainly on creating, optimizing, and using domain-specific languages.

Metacompilers

Complementing language-oriented programming, and all other forms of domain-specific languages, are the class of compiler writing tools called metacompilers. Such compiler is useful for generating parsers and code generators for domain-specific languages, and a metacompiler compiles a domain-specific metalanguage specifically designed for the domain of metaprogramming.

Besides parsing domain-specific languages, metacompilers are useful for generating a wide range of software engineering and analysis tools. The meta-compiler methodology is often found in program transformation systems.

Metacompilers that played a significant role in both computer science and the computer industry include META II, and its descendant TreeMeta.

Unreal Engine before version 4 and other games

Unreal and Unreal Tournament unveiled a language named UnrealScript. This allowed rapid development of modifications relative to the competitor Quake (using the Id Tech 2 engine). The Id Tech engine used standard C code meaning C had to be learned and properly applied, while UnrealScript was optimized for ease of use and efficiency. Similarly, more recent games have introduced their own specific languages for development. One more common example is Lua for scripting.

Rules engines for policy automation

Various business rules engines have been developed for automating policy and business rules used in both government and private industry. ILOG, Oracle Policy Automation, DTRules, Drools and others provide support for DSLs aimed to support various problem domains. DTRules goes so far as to define an interface for the use of multiple DSLs within a rule set.

The purpose of business rules engines is to define a representation of business logic in as human-readable fashion as possible. This allows both subject-matter experts and developers to work with and understand the same representation of the business logic. Most rules engines provide both an approach to simplifying the control structures for business logic (for example, using declarative rules or decision tables) coupled with alternatives to programming syntax in favor of DSLs.

Statistical modelling languages

Statistical modelers have developed domain-specific languages such as R (an implementation of the S language), Bugs, Jags, and Stan. These languages provide a syntax for describing a Bayesian model and generate a method for solving it using simulation.

Generate model and services to multiple programming Languages

Generate object handling and services based on an Interface Description Language for a domain-specific language such as JavaScript for web applications, HTML for documentation, C++ for high-performance code, etc. This is done by cross-language frameworks such as Apache Thrift or Google Protocol Buffers.

Gherkin

Gherkin is a language designed to define test cases to check the behavior of software, without specifying how that behavior is implemented. It is meant to be read and used by non-technical users using a natural language syntax and a line-oriented design. The tests defined with Gherkin must then be implemented in a general programming language. Then, the steps in a Gherkin program acts as a syntax for method invocation accessible to non-developers.

Other examples

Other prominent examples of domain-specific languages include:

• Game Description Language
• OpenGL Shading Language
• Gradle
• ActionScript

Advantages and disadvantages

Some of the advantages:

• Domain-specific languages allow solutions to be expressed in the idiom and at the level of abstraction of the problem domain. The idea is that domain experts themselves may understand, validate, modify, and often even develop domain-specific language programs. However, this is seldom the case.
• Domain-specific languages allow validation at the domain level. As long as the language constructs are safe any sentence written with them can be considered safe.
• Domain-specific languages can help to shift the development of business information systems from traditional software developers to the typically larger group of domain-experts who (despite having less technical expertise) have a deeper knowledge of the domain.
• Domain-specific languages are easier to learn, given their limited scope.

Some of the disadvantages:

• Cost of learning a new language
• Limited applicability
• Cost of designing, implementing, and maintaining a language and the tools needed to develop with it: integrated development environment (IDE)
• Finding, setting, and maintaining proper scope.
• Difficulty of balancing trade-offs between domain-specificity and general-purpose programming language constructs
• Potential loss of processor efficiency compared with hand-coded software
• Proliferation of similar non-standard domain-specific languages, for example, a DSL used in one insurance company versus a DSL used in another insurance company.
• Non-technical domain experts can find it hard to write or modify DSL programs by themselves
• Increased difficulty of integrating the DSL with other components of the IT system; relative to integrating with a general-purpose language
• Low supply of experts in a specific DSL tends to raise labor costs
• Harder to find code examples

Tools for designing domain-specific languages

• JetBrains MPS is a tool for designing domain-specific languages. It uses projectional editing which allows overcoming the limits of language parsers and building DSL editors, such as ones with tables and diagrams. It implements language-oriented programming. MPS combines an environment for language definition, a language workbench, and an integrated development environment (IDE) for such languages.
• MontiCore is a language workbench for the efficient development of domain-specific languages. It processes an extended grammar format that defines the DSL and generates Java components for processing the DSL documents.
• Xtext is an open-source software framework for developing programming languages and domain-specific languages (DSLs). Unlike standard parser generators, Xtext generates not only a parser but also a class model for the abstract syntax tree. In addition, it provides a fully featured, customizable Eclipse-based IDE. The project was archived in April 2023.
• Racket is a cross-platform language toolchain including native code, JIT and JavaScript compiler, IDE (in addition to supporting Emacs, Vim, VSCode and others) and command line tools designed to accommodate creating both domain-specific and general purpose languages.