SQL Data Query

SQL Data Query

SQL Data Query

DLH.io documentation for SQL Data Query

Many of our business customers solve their one off data sharing needs by using a simple sql data query connection.

A SQL Data Query connection allows a SQL query to query data from their source system, such as a Cloud Data Warehouse database and send the results of that data set somewhere else such as via SFTP as part of their integration workflow.

It is as easy as it sounds. Create SQL query against a known data source such as your database table(s), joins are allowed, etc. Then take the proven query and enter it in the SQL Query field of the SQL Data Query.

This now becomes a SQL data query connector that is based one of our existing cloud data warehouse connectors (other application systems such as NetSuite SuiteQL are possible), and it can then be added as a source in your Sync Bridge to load to your target destination of choice, i.e.: Cloud Data Warehouse, SFTP.

Prerequisites

You need to make sure that:

  • You have a source connector already created for your Cloud Data Warehouse

    • SQL data Query connector needs a source from which to query. This is done by selecting the source in question in the new SQL data Query source connection form. Without an existing Cloud Data Warehouse connection (e.g.: Snowflake Source Conector) you will not be able to use this connector.
  • You have an existing working query that you've validated returns results/records.

  • You understand that SQL Data Query results can be limited based on your current subscription plan. The lower plan(s) are limited and controlled to allow a maximum of 100,000 results/records based on any one query.

  • Only one SQL query per SQL Data Query connector are allowed otherwise your connector is likely to result in failure. CTE's are permissable.

Setup Instructions

Using the source connector form please complete the following basic steps:

  1. Enter a Name or Alias for this connection, in the Name/Alias field, that is unique from other connectors.
  2. Enter in the Target Schema Prefix field, a name that will be unique in your data cloud destination where you wish to land the data.
  3. Select from the Source Cloud Database dropdown, a source connection you've previously created from which this query will be executed.
    • If you do not yet have a source connector to a cloud data warehouse you must create one before this connector can be create.
    • image
  4. Enter the Output File Name for what the target result of this query will be referrenced as when it is used by the target system, which can differ slightly depending on the target connection is aligned with in a Sync Bridge for example.
    • JINJA Usage:
      • Basic JINJA can be used for timestamp logic in the folder path to determine what the dynamic structure of the S3 bucket might be, for example as a time based solution, if looking in a folder prefix/bucket as such, 2025/11/10/myfiles/otherfiles/, and the date structure is correctly used over time, then the following JINJA expression can be used in the Folder Path field, {{ today() | format_date('YYYY') }}/{{ today() | format_date('MM') }}/, to ensure that DLH.io is dynamically retrieving from that folder each day
  5. Enter in the SQL Query field the SQL code which you have previously validated
  6. Click Save & Test to save the connector

This article was sourced from DLH.io Documentation.

    • Related Articles

    • SQL Playground Editor

      SQL Playground Editor DLH.io documentation for SQL Playground Editor As a all-in-one data orchestration platform it is often necessary to run basic queries against certain data connectors to confirm logic and data. This can also be used to validate ...
    • What is a Data Warehouse?

      Learn The Basic Concepts What is a Data Warehouse? DLH.io documentation for What is a Data Warehouse? All true business analysts or executive management leadership that has worked effectively with data in the last two decades understands that a ...
    • What Data Sources can I Synchronize to a Target Destination?

      What Data Sources can I Synchronize to a Target Destination? DLH.io currently offers several Connectors (aka connections or integrations) for many popular SaaS, database, and operational source systems, for example: Aloha POS Dayforce Databricks ...
    • Data Warehouse Clouds

      Data Warehouse Clouds DLH.io documentation for Data Warehouse Clouds Cloud Data Warehousing is a means for an organization to store and retrieve curated business decision making information, quickly, accurately, and with high levels of confidence, ...
    • What is a Data Catalog?

      Learn The Basic Concepts What is a Data Catalog? DLH.io documentation for What is a Data Catalog? A Data Catalog, also known as a Metadata Dictionary, or a Data Glossary, is a system which provides your data workers and business analytics users with ...