Introduction

This article outlines how to automate the loading of a JSON response into Oracle BI Cloud Service (BICS) via Oracle Application Express (Oracle Apex).

The solution is PL/SQL based and can also be used for non-BICS database environments such as Oracle DbaaS and Oracle Database Schema Service.

The main components of the PL/SQL code are as follows:

1) Read in the JSON response as a CLOB using the apex_web_service.make_rest_request function.

2) Convert the CLOB to a JSON_LIST data type.

3) Loop through rows of the JSON_LIST.

4) Use the JSON_EXT.GET_STRING function to parse out the JSON attributes.

5) Insert the JSON attributes into a relational table located in the BICS schema.

The rationale for the article is to overcome the following constraints:

1) Currently the REST API Oracle Business Intelligence Cloud Service can not read the JSON structure depicted in the article. This makes it difficult to achieve automated cloud-to-cloud data transfers without additional file conversions.

2) The APEX_JSON package is only available in Oracle Apex 5.0 and higher. At the time this article was written, BICS was shipped with Oracle Apex v4.2.6.00.03. In the future, should the BICS Apex version be upgraded, it may be more preferable to use the functionality available in the APEX_JSON package rather than that in the PL/JSON package suggested in this article.

This article is intended for developers looking for a simple straightforward method to read JSON into BICS using PL/SQL. The primary requirements are:

1) That the JSON response is accessible via a URL.

2) The JSON response can be interpreted by the JSON_LIST data type installed with the PL/JSON package.

The JSON response may originate from a source such as (but not limited to) a mobile device, a website or another cloud application.

The sample code supplied parses the JSON data to a relational database table – making it readily available to model in BICS. Oracle Database 12c has built in JSON functions that allows JSON data to be stored in its original format. This article does not cover these features. However, pending how the data will be consumed, this may be something worth considering.

It is recommended to follow the steps below in chronological order. This will facilitate successful replication and assist with debugging.

Step One – Build the PL/SQL to return the JSON Response

Step Two – Load the PL/JSON Package

Step Three – Create the table to store the data

Step Four – Build the PL/SQL to parse the JSON and insert into the database

Step Five – Merge the code from Step One and Step Four

Main Article

Step One – Build the PL/SQL to return the JSON Response.

# 1-3: Explains how to build the PL/SQL

# 4-5: Describes how to run the PL/SQL – through Oracle Apex SQL Workshop

# 6-7: Illustrates a sample JSON response

1) Replace the following values (that are highlighted in yellow in the below code sample box).

a) l_ws_url – replace with JSON request URL

b) g_request_headers – replace with JSON request headers

2) Confirm that the PL/SQL returns the JSON response successfully.

3) For a text version of this sql code click here

DECLARE
l_ws_response_clob CLOB;
l_ws_url VARCHAR2(500) := ‘https://YourURL‘;
BEGIN
apex_web_service.g_request_headers(1).name := ‘Accept';
apex_web_service.g_request_headers(1).value := ‘application/json; charset=utf-8′;
apex_web_service.g_request_headers(2).name := ‘Content-Type';
apex_web_service.g_request_headers(2).value := ‘application/json; charset=utf-8′;
l_ws_response_clob := apex_web_service.make_rest_request
(
p_url => l_ws_url,
p_http_method => ‘GET’
);
dbms_output.put_line(dbms_lob.substr(l_ws_response_clob,24000,1));
dbms_output.put_line(dbms_lob.substr(l_ws_response_clob,24001,24000));
END;

4) From Oracle APEX -> SQL Workshop

5) Copy the SQL into the SQL Command window and Run -> Confirm that the Results return the JSON response.

6) Below is an example JSON response.

[{"avatarDefinitionId":"13","attributes":{"Name":"HP Jet Pro Wireless","Status":"1","Manufacturer":"HP","Model":"P1102W","Type":"LaserJet"},"lastUpdatedTime":"2015-07-28T12:01:55.919Z"},{"avatarDefinitionId":"13","attributes":{"Name":"Epson Stylus Personal Printer","Status":"1","Manufacturer":"Epson","Model":"C88","Type":"Inkjet"},"lastUpdatedTime":"2015-07-28T12:01:00.000Z"},{"avatarDefinitionId":"13","attributes":{"Name":"Canon PIXMA","Status":"1","Manufacturer":"Cannon","Model":"MG3520","Type":"Wireless All-in-One "},"lastUpdatedTime":"2015-07-28T12:00:21.923Z"},{"avatarDefinitionId":"13","attributes":{"Name":"Konica Minolta Bizhub","Status":"4","Manufacturer":"Konica","Model":"C360","Type":"Color Copier"},"lastUpdatedTime":"2015-07-28T11:51:57.993Z"}]

7) For a text version of the JSON response click here

Step Two – Load the PL/JSON Package

Step Two describes how to load the artifacts from the PL/JSON Package into the BICS Schema.

The BICS schema name is visible from Oracle Apex SQL Workshop (in the Schema drop-down – top right).

It can also be gathered by running the SQL below.

The scripts should be ran by a user the “Database Administrator” role.

Click here for more information on BICS roles.

SELECT
sys_context(‘USERENV’, ‘CURRENT_SCHEMA’) CURRENT_SCHEMA
FROM dual;

1) Download the PL/JSON Package from here

2) Click Download Zip

3) Extract pljson-master.zip

4) The extracted file contains the following:

5) Either run install.sql or run each of the below scripts individually in the order they appear in install.sql

json_value.typ
json_list.typ
json.typ
json_parser.sql
json_printer.sql
json_value_body.typ
json_ext.sql
json_body.typ
json_list_body.typ

6) If loading the scripts through APEX, use the SQL Scripts loader.

SQL Workshop -> SQL Scripts -> Upload -> Run

7) If running the install.sql though SQLDeveloper, change the “select default path to look for scripts” to match where the zip file was extracted.

Tools -> Preferences -> Database -> Worksheet

8) Confirm all objects have a status of VALID

Step Three – Create the table to store the data

CREATE TABLE PRINTER_INFO
(
RECORD_NUM number,
NAME varchar(100),
STATUS varchar(100),
MANUFACTURER varchar(100),
MODEL varchar(100)
);

Step Four – Build the PL/SQL to parse the JSON and insert into the database

1) Save the JSON output from Step One to a local text file.

Download it from here

Rename the file json.txt

Save it to C:\Temp

2) From Apex go to SQL Workshop -> RESTful Services

3) Create a RESTful Service Module.

Name: Printer.info

URI Prefix: Printer/

4) Add a URI Template

URI Template: PostStatus/

5) Add a Resource Handler

Method: POST

Source Type: PL/SQL

Source: Copy SQL from below (#7)

6) Modify the highlighted SQL to match the JSON file and table column names.

Add additional columns and change attribute names where needed.

7) For a text version of this SQL code click here

declare
l_clob clob;
l_warning varchar2(32767);
l_list json_list;
l_dest_offset   integer;
l_src_offset    integer;
l_lang_context integer;
l_col1 VARCHAR2(100);
l_col2 VARCHAR2(100);
l_col3 VARCHAR2(100);
l_col4 VARCHAR2(100);
begin
l_dest_offset := 1;
l_src_offset := 1;
l_lang_context := dbms_lob.default_lang_ctx;
DBMS_LOB.createtemporary(l_clob, FALSE);
DBMS_LOB.CONVERTTOCLOB(
dest_lob       => l_clob,
src_blob       => :body,
amount         => DBMS_LOB.LOBMAXSIZE,
dest_offset    => l_dest_offset,
src_offset     => l_src_offset,
blob_csid      => dbms_lob.default_csid ,
lang_context   => l_lang_context,
warning        => l_warning
);
l_list := json_list(l_clob);
for i in 1..l_list.count LOOP
l_col1   := json_ext.get_string(json(l_list.get(i)),’attributes.Name’);
l_col2   := json_ext.get_string(json(l_list.get(i)),’attributes.Status’);
l_col3   := json_ext.get_string(json(l_list.get(i)),’attributes.Manufacturer’);
l_col4   := json_ext.get_string(json(l_list.get(i)),’attributes.Model’);
INSERT INTO PRINTER_INFO(RECORD_NUM,Name,Status,Manufacturer,Model) VALUES (i,l_col1,l_col2,l_col3,l_col4);
end loop;
end;

8) *** Apply Changes ***

9) Run through curl to test. Download curl from here.

Replace the following highlighted items:

username:password

-k with the Apex URL

-d with the path to the JSON file saved locally (as defined in Step Four – #1)

10) For a text version of the curl command click here

curl -u username:password -X POST -v -k “https://mytrial123-us123.db.usa.oracle.com/apex/Printer/PostStatus/” -H “Content-Type: application/json” -d@C:\temp\json.txt

11) Confirm data loaded correctly

select * from PRINTER_INFO;

Step Five – Merge the code from Step One and Step Four

1) Edit the Resource Handler Source. If preferred, this may also be ran in SQL Workshop.

To get to the Resource Handler Source go to:

SQL Developer -> RESTful Services -> Module -> URI Template -> POST SOURCE

2) Combine the PL/SQL developed in steps one and four. Replace the l_ws_url, g_request_headers, JSON attributes, and insert table columns.

3) For a text version of the PL/SQL click here

declare
l_ws_response_clob CLOB;
l_ws_url VARCHAR2(500) := ‘YourURL';
l_list json_list;
l_col1 VARCHAR2(100);
l_col2 VARCHAR2(100);
l_col3 VARCHAR2(100);
l_col4 VARCHAR2(100);
begin
–get JSON
apex_web_service.g_request_headers(1).name := ‘Accept';
apex_web_service.g_request_headers(1).value := ‘application/json; charset=utf-8′;
apex_web_service.g_request_headers(2).name := ‘Content-Type';
apex_web_service.g_request_headers(2).value := ‘application/json; charset=utf-8′;
l_ws_response_clob := apex_web_service.make_rest_request(
p_url => l_ws_url,
p_http_method => ‘GET’
);
–dbms_output.put_line(dbms_lob.substr(l_ws_response_clob,24000,1));
–dbms_output.put_line(dbms_lob.substr(l_ws_response_clob,24001,24000));
–convert clob to json_list
l_list := json_list(l_ws_response_clob);
for i in 1..l_list.count LOOP
l_col1   := json_ext.get_string(json(l_list.get(i)),’attributes.Name’);
l_col2   := json_ext.get_string(json(l_list.get(i)),’attributes.Status’);
l_col3   := json_ext.get_string(json(l_list.get(i)),’attributes.Manufacturer’);
l_col4   := json_ext.get_string(json(l_list.get(i)),’attributes.Model’);
INSERT INTO PRINTER_INFO(RECORD_NUM,Name,Status,Manufacturer,Model) VALUES (i,l_col1,l_col2,l_col3,l_col4);
end loop;
end;

4) Apply Changes

5) Delete the records that were previously loaded

Delete from PRINTER_INFO;

6) Re-run through curl. This time omitting the -H and -d flags.

For a text version of the curl command click here

curl -u username:password -X POST -v -k “https://mytrial123-us123.db.usa.oracle.com/apex/Printer/PostStatus/”

7) Confirm data loaded successfully

select * from PRINTER_INFO;

Summary

This article describes how to integrate JSON with BICS. The solution requires the installation of the referenced PL/JSON package and the code is accessed through the APEX RESTful Services. That said, the APEX RESTful Services are not mandatory and other technologies more appropriate to the individual use case can be utilized. For illustration purposes Curl has been used to transfer the data. It is anticipated in a production environment alternative data transfer methods may be preferred. This article was written to appeal to a BICS’s audience. Additionally, Non-BICS users may also find it beneficial in particular those consuming JSON through cloud or mobile technologies wanting to load the data into Oracle DbaaS or Oracle Database Schema Service (via Oracle APEX).

All content listed on this page is the property of Oracle Corp. Redistribution not allowed without written permission