DataFlex
DataFlex
Markdown to HTML - Streaming Mode
See more Markdown Examples
Shows how to convert Markdown to HTML in streaming mode, ideal for processing deltas with fragments of incoming Markdown during AI streaming responses. This example will simulate processing incoming AI delta's by streaming 80-byte chunks from a markdown file.Chilkat DataFlex Downloads
Use ChilkatAx-win32.pkg
Procedure Test
Boolean iSuccess
Variant vOptions
Handle hoOptions
Handle hoSbFullMarkdown
Frag Handle hoSbHtmlFrag
Handle hoSbHtml
Handle hoSbStreamingMarkdown
Handle hoSbMdChunk
Integer iSafety
Integer iChunkSize
String s
String sTemp1
Integer iTemp1
Move False To iSuccess
Get Create (RefClass(cComChilkatJsonObject)) To hoOptions
If (Not(IsComObjectCreated(hoOptions))) Begin
Send CreateComObject of hoOptions
End
Get ComUpdateString Of hoOptions "theme" "raw" To iSuccess
// Indicate streaming mode
Get ComUpdateBool Of hoOptions "streaming" True To iSuccess
// Load the full markdown file, which will be converted to HTML 80 bytes at a time in streaming mode.
// The sample markdown input for this example is identical to the one at Markdown to HTML - Full Document, Raw, where you can view it.
Get Create (RefClass(cComChilkatStringBuilder)) To hoSbFullMarkdown
If (Not(IsComObjectCreated(hoSbFullMarkdown))) Begin
Send CreateComObject of hoSbFullMarkdown
End
Get ComLoadFile Of hoSbFullMarkdown "qa_data/markdown/test1.md" "utf-8" To iSuccess
If (iSuccess = False) Begin
Get ComLastErrorText Of hoSbFullMarkdown To sTemp1
Showln sTemp1
Procedure_Return
End
// This will contain the fragment of HTMl produced at each step in streaming mode.
Get Create (RefClass(cComChilkatStringBuilder)) To hoSbHtmlFrag
If (Not(IsComObjectCreated(hoSbHtmlFrag))) Begin
Send CreateComObject of hoSbHtmlFrag
End
// We'll accumulate the full HTML result here, by appending each HTLM fragment
Get Create (RefClass(cComChilkatStringBuilder)) To hoSbHtml
If (Not(IsComObjectCreated(hoSbHtml))) Begin
Send CreateComObject of hoSbHtml
End
// This will contain the chunk of markdown not yet converted to HTML.
Get Create (RefClass(cComChilkatStringBuilder)) To hoSbStreamingMarkdown
If (Not(IsComObjectCreated(hoSbStreamingMarkdown))) Begin
Send CreateComObject of hoSbStreamingMarkdown
End
// This contains the current 80 byte markdown chunk to be processed.
// The last chunk can be less than 80 bytes.
Get Create (RefClass(cComChilkatStringBuilder)) To hoSbMdChunk
If (Not(IsComObjectCreated(hoSbMdChunk))) Begin
Send CreateComObject of hoSbMdChunk
End
// Let's safeguard the loop to process a max of 160K of markdown (2000 x 80 bytes = 160K)
// This is to prevent an infinite loop in case something isn't working correctly.
Move 0 To iSafety
While (((ComLength(hoSbFullMarkdown)) > 0) And (iSafety < 2000))
Move 80 To iChunkSize
Get ComLength Of hoSbFullMarkdown To iTemp1
If (iTemp1 < iChunkSize) Begin
Get ComLength Of hoSbFullMarkdown To iChunkSize
End
// Remove the leading chunk from the sbFullMarkdown and append to sbStreamingMarkdown.
Get ComGetRange Of hoSbFullMarkdown 0 iChunkSize True To s
Get ComAppend Of hoSbStreamingMarkdown s To iSuccess
Showln "-------------------------------------------------------------"
Get ComToCRLF Of hoSbStreamingMarkdown To iSuccess
Get ComGetAsString Of hoSbStreamingMarkdown To sTemp1
Showln sTemp1
// Convert what is possible and append to the HTML fragment.
// It's possible that no additional HTML can be generated if the markdown chunk is does not contain even a single full line.
// In streaming mode, MarkdownToHtml appends any additional HTML it can generate to the StringBuilder passed in the 2nd argument.
// If we want to only see what new HTML was generated from this chunk, we need to clear the StringBuilder before calling MarkdownToHtml.
// The markdown that was converted is removed from the caller's contents.
Send ComClear To hoSbHtmlFrag
Get pvComObject of hoOptions to vOptions
Get pvComObject of hoSbHtmlFrag to vSbHtmlFrag
Get ComMarkdownToHtml Of hoSbStreamingMarkdown vOptions vSbHtmlFrag To iSuccess
// Accumulate the HTML fragments into a full HTML document.
Get pvComObject of hoSbHtmlFrag to vSbHtmlFrag
Get ComAppendSb Of hoSbHtml vSbHtmlFrag To iSuccess
Showln "----"
Get ComToCRLF Of hoSbHtmlFrag To iSuccess
Get ComGetAsString Of hoSbHtmlFrag To sTemp1
Showln sTemp1
Move (iSafety + 1) To iSafety
Loop
// Flush any remaining closing HTML tags by passing a final line-ending.
Showln "----FINAL----------------------------------------------------"
Get ComAppend Of hoSbStreamingMarkdown (character(10)) To iSuccess
Get ComGetAsString Of hoSbStreamingMarkdown To sTemp1
Showln sTemp1
Send ComClear To hoSbHtmlFrag
Get pvComObject of hoOptions to vOptions
Get pvComObject of hoSbHtmlFrag to vSbHtmlFrag
Get ComMarkdownToHtml Of hoSbStreamingMarkdown vOptions vSbHtmlFrag To iSuccess
Get pvComObject of hoSbHtmlFrag to vSbHtmlFrag
Get ComAppendSb Of hoSbHtml vSbHtmlFrag To iSuccess
Showln "----"
Get ComGetAsString Of hoSbHtmlFrag To sTemp1
Showln sTemp1
Showln ""
Showln "**** Full Accumulated HTML ****"
Get ComToCRLF Of hoSbHtml To iSuccess
Get ComGetAsString Of hoSbHtml To sTemp1
Showln sTemp1
// Notice how the 1st loop iteration produces no HTML output. This is because there is not yet a complete line of markdown to process.
// Sample ouput:
// -------------------------------------------------------------
// Here’s a simple implementation of the standard C library function `strncpy`, wri
// ----
//
// -------------------------------------------------------------
// Here’s a simple implementation of the standard C library function `strncpy`, written from scratch — demonstrating how it works internally:
//
// ```c
// #include <st
// ----
// <p>Here’s a simple implementation of the standard C library function <code>strncpy</code>, written from scratch — demonstrating how it works internally:</p>
//
// -------------------------------------------------------------
// #include <stddef.h> // for size_t
//
// char *my_strncpy(char *dest, const char *src, size_t n
// ----
// <pre id="pre_dd232b5f-2279-4841-b602-5840072f7a84"><code id="dd232b5f-2279-4841-b602-5840072f7a84" class="language-c">#include <stddef.h> // for size_t
//
//
// -------------------------------------------------------------
// char *my_strncpy(char *dest, const char *src, size_t n)
// {
// size_t i;
//
// for (i = 0; i < n && src[i] != '\0'; i++) {
//
// ----
// char *my_strncpy(char *dest, const char *src, size_t n)
// {
// size_t i;
//
// for (i = 0; i < n && src[i] != '\0'; i++) {
//
// -------------------------------------------------------------
// dest[i] = src[i];
// }
//
// // If src is shorter than n, pad with '\0'
//
// ----
// dest[i] = src[i];
// }
//
// // If src is shorter than n, pad with '\0'
//
// -------------------------------------------------------------
// for (; i < n; i++) {
// dest[i] = '\0';
// }
//
// return dest;
// }
// ```
// ----
// for (; i < n; i++) {
// dest[i] = '\0';
// }
//
// return dest;
// }
//
// -------------------------------------------------------------
// ```
//
// ### Explanation:
//
// * **Parameters:**
//
// * `dest`: destination buffer to c
// ----
// </code></pre>
// <h3>Explanation:</h3>
// <ul>
// <li><strong>Parameters:</strong>
// -------------------------------------------------------------
// * `dest`: destination buffer to copy into.
// * `src`: source string.
// * `n`: maximum number of bytes to copy.
//
// ----
// <ul>
// <li><code>dest</code>: destination buffer to copy into.</li>
// <li><code>src</code>: source string.</li>
// <li><code>n</code>: maximum number of bytes to copy.
// -------------------------------------------------------------
//
//
// * **Logic:**
//
// 1. Copy characters from `src` into `dest` until you hit eit
// ----
// </li>
// </ul>
// </li>
// <li><strong>Logic:</strong>
// -------------------------------------------------------------
// 1. Copy characters from `src` into `dest` until you hit either:
//
// * The null terminator (`'\0'`) in `src`, or
// * The limit `n`.
//
// ----
// <ol>
// <li>Copy characters from <code>src</code> into <code>dest</code> until you hit either:
// <ul>
// <li>The null terminator (<code>'\0'</code>) in <code>src</code>, or</li>
// <li>The limit <code>n</code>.
// -------------------------------------------------------------
//
// 2. If you reached the end of `src` before hitting `n`, pad the remaining byte
// ----
//
// -------------------------------------------------------------
// 2. If you reached the end of `src` before hitting `n`, pad the remaining bytes of `dest` with `'\0'`.
// 3. Return `dest` so it behaves like standard C libra
// ----
// </li>
// </ul>
// </li>
// <li>If you reached the end of <code>src</code> before hitting <code>n</code>, pad the remaining bytes of <code>dest</code> with <code>'\0'</code>.
// -------------------------------------------------------------
// 3. Return `dest` so it behaves like standard C library functions.
//
// ### Notes:
//
// * `strncpy` does **not** guarantee null-terminati
// ----
// </li>
// <li>Return <code>dest</code> so it behaves like standard C library functions.</li>
// </ol>
// </li>
// </ul>
// <h3>Notes:</h3>
//
// -------------------------------------------------------------
// * `strncpy` does **not** guarantee null-termination if `src` has length ≥ `n`.
// To ensure a null-terminated string, you can do:
// ----
// <ul>
// <li><code>strncpy</code> does <strong>not</strong> guarantee null-termination if <code>src</code> has length ≥ <code>n</code>.
// -------------------------------------------------------------
// To ensure a null-terminated string, you can do:
//
// ```c
// dest[n - 1] = '\0';
// ```
//
// — but only if you *always* reserve
// ----
//
// To ensure a null-terminated string, you can do:
// <pre id="pre_80ca4db3-f745-473c-91d7-a565d106bac8"><code id="80ca4db3-f745-473c-91d7-a565d106bac8" class="language-c">dest[n - 1] = '\0';
// </code></pre>
//
// -------------------------------------------------------------
// — but only if you *always* reserve space for that final null.
//
// ---
//
// Would you like me to also show a **safer**
// ----
// — but only if you <em>always</em> reserve space for that final null.</li>
// </ul>
// <hr />
//
// -------------------------------------------------------------
// Would you like me to also show a **safer** version (like how `strlcpy` behaves) that guarantees null-termination?
//
// ----
// <p>Would you like me to also show a <strong>safer</strong> version (like how <code>strlcpy</code> behaves) that guarantees null-termination?
// ----FINAL----------------------------------------------------
//
//
// ----
// </p>
// *******************************
// **** Full Accumulated HTML ****
// *******************************
// <p>Here’s a simple implementation of the standard C library function <code>strncpy</code>, written from scratch — demonstrating how it works internally:</p>
// <pre id="pre_dd232b5f-2279-4841-b602-5840072f7a84"><code id="dd232b5f-2279-4841-b602-5840072f7a84" class="language-c">#include <stddef.h> // for size_t
//
// char *my_strncpy(char *dest, const char *src, size_t n)
// {
// size_t i;
//
// for (i = 0; i < n && src[i] != '\0'; i++) {
// dest[i] = src[i];
// }
//
// // If src is shorter than n, pad with '\0'
// for (; i < n; i++) {
// dest[i] = '\0';
// }
//
// return dest;
// }
// </code></pre>
// <h3>Explanation:</h3>
// <ul>
// <li><strong>Parameters:</strong><ul>
// <li><code>dest</code>: destination buffer to copy into.</li>
// <li><code>src</code>: source string.</li>
// <li><code>n</code>: maximum number of bytes to copy.</li>
// </ul>
// </li>
// <li><strong>Logic:</strong><ol>
// <li>Copy characters from <code>src</code> into <code>dest</code> until you hit either:
// <ul>
// <li>The null terminator (<code>'\0'</code>) in <code>src</code>, or</li>
// <li>The limit <code>n</code>.</li>
// </ul>
// </li>
// <li>If you reached the end of <code>src</code> before hitting <code>n</code>, pad the remaining bytes of <code>dest</code> with <code>'\0'</code>.</li>
// <li>Return <code>dest</code> so it behaves like standard C library functions.</li>
// </ol>
// </li>
// </ul>
// <h3>Notes:</h3>
// <ul>
// <li><code>strncpy</code> does <strong>not</strong> guarantee null-termination if <code>src</code> has length ≥ <code>n</code>.
// To ensure a null-terminated string, you can do:
// <pre id="pre_80ca4db3-f745-473c-91d7-a565d106bac8"><code id="80ca4db3-f745-473c-91d7-a565d106bac8" class="language-c">dest[n - 1] = '\0';
// </code></pre>
// — but only if you <em>always</em> reserve space for that final null.</li>
// </ul>
// <hr />
// <p>Would you like me to also show a <strong>safer</strong> version (like how <code>strlcpy</code> behaves) that
End_Procedure