Sharp Scripts

Unlike most languages, #Script has 2 outputs, the side-effect of the script itself and its textual output which makes it ideal for literate programming where executable snippets can be embedded inside an executable document.

code blocks are a convenient markup for embedding executable scripts within a document without the distracting boilerplate of wrapping each statement in expression blocks. Together with x watch, #Script allows for an iterative, exploratory style of programming in a live programming environment that benefits many domains whose instant progressive feedback unleashes an unprecedented amount of productivity, one of those areas it benefits is shell scripting where the iterative feedback is invaluable in working towards a solution for automating the desired task.

Some protected script methods used to improve Usage in shell scripts, include:

string proc(string exeFileName, {arguments:string, dir:string})
- Execute a local system process
  - arguments: command line args
  - dir: working directory
sh(cmdArgs, {dir:string})
- Executes shell commands. Uses cmd.exe /C {cmdArgs} in Windows, otherwise /bin/bash -c {cmdArgs} on macOS/Linux
string exePath(string exeName)
- Returns the full path to an executable located in the users $PATH
string osPaths(string path)
- Rewrite paths to use \ for Windows, otherwise uses /

From this list, you're likely to use sh the most to execute arbitrary commands with:

cmd |> sh

As with any #Script method, it can also be executed using the less natural forms of cmd.sh() or sh(cmd)

Which will emit the StandardOutput of the command if successful, otherwise it throws an Exception if anything was written to the StandardError. Checkout the Error Handling docs for how to handle exceptions.

With the features in this release, #Script has now became our preferred way to create x-plat shell scripts which can be run with the Windows app or x-plat x dotnet tool which runs everywhere .NET Core does.

As such, all scripts in ServiceStack/mix provides a good collection of .ss scripts to check out which are used to maintain the Gist content that powers the new mix feature.

Live Shell Scripting Preview

To provide a glimpse of the productivity of this real-time exploratory programming-style, here's an example of a task we needed to automate to import and rename 48px SVG images from Material Design Icons Project folders into the form needed in the mix Gists:

YouTube: youtu.be/joXSHtfb_7g

Here we can see how we're able to progressively arrive at our solution without leaving the source code, with the watched script automatically updated at each Ctrl+S save point where we're able to easily infer its behavior from its descriptive textual output.

Explore HTTP APIs in real-time

The real-time scriptability of #Script makes it ideal for a whole host of Exploratory programming tasks that would otherwise be painful in the slow code/build/debug cycles of a compiled language like C#, like being able to cut, sort & slice HTTP APIs with ease:

YouTube: youtu.be/Yjx_9Tp91bQ

Live Querying of Databases

It also serves a great tool for data exploration, akin to a programmable SQL Studio environment with instant feedback:

YouTube: youtu.be/HCjxVJ8RyPc

That benefits from being able to maintain reusable queries in simple, executable text files that can be organized along with other shell scripts and treated like source code where it can be checked into a versionable repo, that anyone will be able to checkout and run from the command-line in Windows, macOS or Linux OS's.

Refer to Sharp App Docs for different db.connection examples of supported RDBMS's.

Utilize high-level ServiceStack Features

Here's another example showing the expressive power of #Script and its comprehensive high-level library which is used to update all library dependencies of the Vue and React "lite" Project Templates:

```code
* Update /libraries gists           *
* Usage: web run libraries.ss <id>? *

{{
    {
        'react-lite-lib': 'ad42adc11337c243ee203f9e9f84622c',
        'vue-lite-lib':   '717258cd4c26ba612e5eed0615d8d61c',
    }
    |> to => gistMap
}}

(ARGV.Length > 0 ? ARGV : gistMap.Keys) |> to => keys

#each id in keys
    gistMap[id] |> to => gistId

    {} |> to => files

    vfsFileSystem(`libraries/${id}`) |> to => fs
    #each file in fs.allFiles()
        files.putItem(file.VirtualPath.replace('/','\\'), file.fileContents()) |> end
    /each

    `Writing to ${files.count()} files to ${id} '${gistId}' ...` |> raw
    vfsGist(gistId, 'GITHUB_GIST_TOKEN'.envVariable()) |> to => gist
    gist.writeFiles(files)
/each
```

Running without any arguments:

$ x run libraries.ss

will update both React and Vue dependencies:

Writing to 26 files to react-lite-lib 'ad42adc11337c243ee203f9e9f84622c' ...
Writing to 41 files to vue-lite-lib '717258cd4c26ba612e5eed0615d8d61c' ...

Alternatively it can be limited to updating a single Framework dependencies with:

$ x run libraries.ss vue-lite-lib

The web dotnet tool also includes the capability of both executing #Script scripts as well as watching scripts to enable a live visual REPL with instant real-time feedback that makes it perfect for Exploratory tasks.

Bundling and Minification

The Vue and React "lite" project templates take advantage of this in their Pre-compiled minified production _bundle.ss script which is run with x run {script}:

```code
* run in host project directory with `x run wwwroot/_bundle.ss` *

false |> to => debug
(debug ? '' : '.min')       |> to => min
(debug ? '' : '[hash].min') |> to => dist

{{ [`/css/lib.bundle${dist}.css`,`/js/lib.bundle${dist}.js`,`/js/bundle${dist}.js`] 
   |> map => it.replace('[hash]','.*').findFiles()
   |> flat
   |> do => it.VirtualPath.deleteFile() }}

* Copy same bundle definitions from _layout.html as-is *

['!/assets/css/default.css','/assets/css/'] |> bundleCss({ disk:!debug, out:`/css/lib.bundle${dist}.css` })

{{ [
    `/lib/vue/dist/vue${min}.js`,
    `/lib/vue-router/dist/vue-router${min}.js`,
    '/lib/vue-class-component/vue-class-component.js',
    '/lib/vue-property-decorator/vue-property-decorator.umd.js',
    '/lib/@servicestack/client/servicestack-client.umd.js',
    '/lib/@servicestack/vue/servicestack-vue.umd.js',
] |> bundleJs({ disk:!debug, out:`/js/lib.bundle${dist}.js` }) }}

{{ [
    'content:/src/components/',
    'content:/src/shared/',
    'content:/src/',
] |> bundleJs({ minify:!debug, cache:!debug, disk:!debug, out:`/js/bundle${dist}.js`, iife:true }) }}
```

Which can be run with the x tool:

$ dotnet tool install --global x 

$ x run wwwroot/_bundle.ss

Which will create the production bundles, minify all already non-minified bundles and write them to disk with the paths written visible in the #Script *_bundle.ss* output:

<link rel="stylesheet" href="/css/bundle.min.css">
    
<script src="/js/lib.bundle.min.js"></script>

<script src="/js/bundle.min.js"></script>

Sharp Scripts run in context of Sharp Apps

Sharp Scripts are run in the same context and have access to the same functionality and features as a Sharp App including extensibility va custom plugins. They can run stand-alone independent of an app.settings config file, instead the app settings configuration can be added in its page arguments to enable or configure any features.

Lets go through a couple of different possibilities we can do with scripts:

AWS Dashboards

The comprehensive built-in scripts coupled with ServiceStack's agnostic providers like the Virtual File System makes it easy to quickly query infrastructure resources like all Tables and Row counts in managed AWS RDS Instances or Search for static Asset resources in S3 Buckets.

<!--
db postgres
db.connection $AWS_RDS_POSTGRES
files s3
files.config {AccessKey:$AWS_S3_ACCESS_KEY,SecretKey:$AWS_S3_SECRET_KEY,Region:us-east-1,Bucket:rockwind}
-->

Querying AWS...

```code
dbTableNamesWithRowCounts |> textDump({ caption: 'Tables' })

5 |> to => limit

`Last ${limit} Orders:\n`
{{ `SELECT * FROM "Order" ORDER BY "Id" DESC ${limit.sqlLimit()}` 
  |> dbSelect |> map => { it.Id, it.CustomerId, it.EmployeeId, Freight: it.Freight.currency() } |> textDump }}

{{ vfsContent.allRootDirectories().map(dir => `${dir.Name}/`) 
  .union(vfsContent.allRootFiles().map(file => file.Name)) |> textDump({caption:'Root Files + Folders'}) }}

(ARGV.first() ?? '*.jpg') |> to => pattern
`\nFirst ${limit} ${pattern} files in S3:`
vfsContent.findFiles(pattern) |> take(limit) |> map => it.VirtualPath |> join('\n')
```

You can use $NAME to move confidential information out of public scripts where it will be replaced with Environment Variables. Then run the script as normal and optionally provide a pattern for files you want to search for:

$ x run script-aws.ss *.jpg

Where it displays a dashboard of activity from your AWS resources: containing all Tables with their Row Counts, adhoc queries like your last 5 Orders, The Root files and Folders available in your S3 Bucket and any matching resources from your specified search pattern:

Querying AWS...

| Tables                   ||
|--------------------|------|
| Order Detail       | 2155 |
| Order              | 830  |
| Customer           | 91   |
| Product            | 77   |
| Territory          | 53   |
| Region             | 0    |
| Shipper            | 0    |
| Supplier           | 0    |
| Category           | 0    |
| Employee           | 0    |
| Employee Territory | 0    |

Last 5 Orders:

| # | Id    | CustomerId | EmployeeId | Freight |
|---|-------|------------|------------|---------|
| 1 | 11077 | RATTC      | 1          | $8.53   |
| 2 | 11076 | BONAP      | 4          | $38.28  |
| 3 | 11075 | RICSU      | 8          | $6.19   |
| 4 | 11074 | SIMOB      | 7          | $18.44  |
| 5 | 11073 | PERIC      | 2          | $24.95  |


| Root Files + Folders   |
|------------------------|
| api/                   |
| northwind/             |
| rockstars/             |
| index.html             |
| web.aws.settings       |
| web.postgres.settings  |
| web.sqlite.settings    |
| web.sqlserver.settings |


First 5 *.jpg files in S3:
assets/img/home-icon.jpg
rockstars/alive/grohl/splash.jpg
rockstars/alive/love/splash.jpg
rockstars/alive/springsteen/splash.jpg
rockstars/alive/vedder/splash.jpg

Azure Dashboards

The nice thing about #Script late-binding and cloud agnostic providers is that with just different configuration we can use the exact same script to query an Azure managed SQL Server Database and Azure Blob File Storage:

<!--
db sqlserver
db.connection $AZURE_SQL_CONNECTION_STRING
files azure
files.config {ConnectionString:$AZURE_BLOB_CONNECTION_STRING,ContainerName:rockwind}
-->

Querying Azure...

```code
dbTableNamesWithRowCounts |> textDump({ caption: 'Tables' })

5 |> to => limit

`Last ${limit} Orders:\n`
{{ `SELECT * FROM "Order" ORDER BY "Id" DESC ${limit.sqlLimit()}` 
  |> dbSelect |> map => { it.Id, it.CustomerId, it.EmployeeId, Freight: it.Freight.currency() } |> textDump }}

{{ vfsContent.allRootDirectories().map(dir => `${dir.Name}/`) 
  .union(vfsContent.allRootFiles().map(file => file.Name)) |> textDump({caption:'Root Files + Folders'}) }}

(ARGV.first() ?? '*.jpg') |> to => pattern
`\nFirst ${limit} ${pattern} files in Azure Blob Storage:`
vfsContent.findFiles(pattern) |> take(limit) |> map => it.VirtualPath |> join('\n')
```

Live #Script with 'web watch'

What's even nicer than the fast feedback of running adhoc scripts? Is the instant feedback you get from being able to "watch" the same script!

To watch a script just replace run with watch:

$ x watch script-aws.ss *.jpg

YouTube: youtu.be/GQvxyPHQjhM

The ability to run stand-alone adhoc scripts in an extensible dynamic scripting language feels like you're using a "developer enhanced" SQL Studio, where you can combine queries from multiple data sources, manipulate them with LINQ and quickly pipe results to dump utils to combine them in the same output for instant visualization.

#Script scripts can also be easily shared, maintained in gists and run on all different Win/OSX/Linux OS's that .NET Core runs on.

Live Transformations

Another area where "watched" scripts can shine is as a "companion scratch pad" assistant during development that you can quickly switch to and instantly test out live code fragments, calculations and transformations, e.g. This ends up being a great way to test out markdown syntax and Nuglify's advanced compression using our new minifyjs and minifycss Script Blocks:

<!--
debug false
-->

Markdown:
{{#markdown}}
## Title

> quote

Paragraph with [a link](https://example.org).
{{/markdown}}

JS:
{{#minifyjs}}
function add(left, right) {
    return left + right;
}
add(1, 2);
{{/minifyjs}}


CSS:
{{#minifycss}}
body {
    background-color: #ffffff;
}
{{/minifycss}}

Then run with:

$ x watch livepad.ss

Which starts a live watched session that re-renders itself on save, initially with:

Markdown:
<h2 id="title">Title</h2>
<blockquote>
<p>quote</p>
</blockquote>
<p>Paragraph with <a href="https://example.org">a link</a>.</p>

JS:
function add(n,t){return n+t}add(1,2)

CSS:
body{background-color:#fff}

Live Session

Adhoc reports

Scripts can use the built-in Database Scripts to be able to run queries against any sqlite, sqlserver, mysql and postgres database and quickly view data snapshots using the built-in HTML Scripts, e.g:

<!--
db sqlite
db.connection ~/../apps/northwind.sqlite
-->

<link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.3.1/css/bootstrap.min.css">
<style>body {padding:1em} caption{caption-side:top;}</style>
<h1 class="py-2">Order Report #{{id}}</h1>

{{ `SELECT o.Id, OrderDate, CustomerId, Freight, e.Id as EmployeeId, s.CompanyName as ShipVia, 
           ShipAddress, ShipCity, ShipPostalCode, ShipCountry
      FROM "Order" o
           INNER JOIN
           Employee e ON o.EmployeeId = e.Id
           INNER JOIN
           Shipper s ON o.ShipVia = s.Id
     WHERE o.Id = @id` 
  |> dbSingle({ id }) |> to => order }}

{{#with order}}
  {{ "table table-striped" |> to => className }}
  <style>table {border: 5px solid transparent} th {white-space: nowrap}</style>
  
  <div style="display:flex">
      {{ order |> htmlDump({ caption: 'Order Details', className }) }}
      {{ `SELECT * FROM Customer WHERE Id = @CustomerId` 
         |> dbSingle({ CustomerId }) |> htmlDump({ caption: `Customer Details`, className }) }}
      {{ `SELECT Id, LastName, FirstName, Title, City, Country, Extension FROM Employee WHERE Id=@EmployeeId` 
         |> dbSingle({ EmployeeId }) |> htmlDump({ caption: `Employee Details`, className }) }}
  </div>

  {{ `SELECT p.ProductName, ${sqlCurrency("od.UnitPrice")} UnitPrice, Quantity, Discount
        FROM OrderDetail od
             INNER JOIN
             Product p ON od.ProductId = p.Id
       WHERE OrderId = @id`
      |> dbSelect({ id }) 
      |> htmlDump({ caption: "Line Items", className }) }}
{{else}}
  {{ `There is no Order with id: ${id}` }}
{{/with}}

Specifying Script Arguments

The above script generates a static HTML page can be invoked with any number of named arguments after the script name, in this case it generates a report for Northwind Order #10643, saves it to 10643.html and opens it in the OS's default browser:

$ x run script.html -id 10643 > 10643.html && start 10643.html

Which looks like:

textDump

Generating static .html pages can quickly produce reports that looks good enough to share with others, but if you just want to see a snapshot info at a glance or be able to share in text-based mediums like email or chat channels you can replace htmlDump with textDump where it will instead output GitHub flavored Markdown tables, e.g:

<!--
db sqlite
db.connection ~/../apps/northwind.sqlite
-->

{{ `SELECT o.Id, OrderDate, CustomerId, Freight, e.Id as EmployeeId, s.CompanyName as ShipVia, 
           ShipAddress, ShipCity, ShipPostalCode, ShipCountry
      FROM "Order" o
           INNER JOIN
           Employee e ON o.EmployeeId = e.Id
           INNER JOIN
           Shipper s ON o.ShipVia = s.Id
     WHERE o.Id = @id` 
  |> dbSingle({ id }) |> to => order }}

{{#with order}}

{{ order |> textDump({ caption: 'Order Details' }) }}

{{ `SELECT p.ProductName, ${sqlCurrency("od.UnitPrice")} UnitPrice, Quantity, Discount
      FROM OrderDetail od
           INNER JOIN
           Product p ON od.ProductId = p.Id
     WHERE OrderId = @id`
    |> dbSelect({ id }) 
    |> textDump({ caption: "Line Items" }) 
}}
{{ `SELECT ${sqlCurrency("(od.UnitPrice * Quantity)")} AS OrderTotals 
      FROM OrderDetail od
           INNER JOIN
           Product p ON od.ProductId = p.Id
     WHERE OrderId = @id 
     ORDER BY 1 DESC`
    |> dbSelect({ id }) 
    |> textDump({ rowNumbers: false }) }}
{{else}}
  {{ `There is no Order with id: ${id}` }}
{{/with}}

As the output is human-readable we can view directly it without a browser:

$ x run script.ss -id 10643

Which will output:

| Order Details                    ||
|------------------|----------------|
| Id               | 10643          |
| Order Date       | 1997-08-25     |
| Customer Id      | ALFKI          |
| Freight          | 29.46          |
| Employee Id      | 6              |
| Ship Via         | Speedy Express |
| Ship Address     | Obere Str. 57  |
| Ship City        | Berlin         |
| Ship Postal Code | 12209          |
| Ship Country     | Germany        |


Line Items

| # | Product Name      | Unit Price | Quantity | Discount |
|---|-------------------|------------|----------|----------|
| 1 | Rössle Sauerkraut | $45.60     | 15       | 0.25     |
| 2 | Chartreuse verte  | $18.00     | 21       | 0.25     |
| 3 | Spegesild         | $12.00     | 2        | 0.25     |


| Order Totals |
|--------------|
| $684.00      |
| $378.00      |
| $24.00       |

And because they're GitHub Flavored Markdown Tables they can be embedded directly in Markdown docs (like this) where it's renders as:

Order Details
Id	10643
Order Date	1997-08-25
Customer Id	ALFKI
Freight	29.46
Employee Id	6
Ship Via	Speedy Express
Ship Address	Obere Str. 57
Ship City	Berlin
Ship Postal Code	12209
Ship Country	Germany

Line Items

#	Product Name	Unit Price	Quantity	Discount
1	Rössle Sauerkraut	$45.60	15	0.25
2	Chartreuse verte	$18.00	21	0.25
3	Spegesild	$12.00	2	0.25

Order Totals
$684.00
$378.00
$24.00

← Sharp APIs Code Pages →