Hadoop Integration Guide for Unica Campaign

Hadoop Integration Guide for Unica Campaign

Big Data. Big Details.

Many of you who attended our webinar on Connecting Hadoop Big Data to Unica Campaign and Interact webinar requested that we provide more detailed technical information about how Unica Campaign was configured to work with Hadoop in the use cases we reviewed. Be forewarned! What follows is a highly technical integration guide of how to configure this type of connection in your environment and is not for the faint of heart.

If you are interested in understanding how Hadoop differs from traditional relational databases from a functional Unica Campaign perspective, you may find our blog on The Facts: Hadoop Big Data vs. Relational Databases more palatable.

Integration Guide Breakdown:

Step 1: Selecting a Hive ODBC Distribution

The first step to connect your Unica Campaign environment to your Apache Hive instance is to download and install a Hive ODBC Driver. Unica Campaign v9.1.2 supports the DataDirect and the Clouder ODBC Drivers, while v10.0 is also to support the Hortonworks driver.

DataDirect Apache Hive ODBC Driver from Progress.com
Progress Logo
Cloudera ODBC Driver for Apache Hive
cloudera Logo
Hortonworks 64-bit ODBC Driver
HortonWorks Logo
Step 2: Installing the Hive ODBC Drive

1. Download the Hive ODBC driver from one of the providers listed above.

2. Upload the RPM file to the server hosting the Campaign Listener (analytics server).

cloudera ODBC installation

3. Download the UnixODBC Manager, unixODBC-2.3.4, available from unixodbc.org.

4. Upload the unixODBC-2.3.4.tar.gz file to the Campaign Listener server.

5. Extract the unixODBC-2.3.4.tar.gz file on your Campaign Listener server.

tar –xvf unixODBC-2.3.4.tar.gz

6. Install the Cloudera Hive driver:

rpm –ivh ClouderaHiveODBC-2.5.16.1005-1.el6.x86_64.rpm

**Note: root permissions may be required.

Step 3: Configure the ODBC.ini file

Once the ODBC driver is installed, you must configure your IBM Campaign server to be able to connect to the Hive distribution. First, you must configure the odbc.ini file which was either included with your Hive ODBC distribution or an existing driver. Edit the odbc.ini file and complete each of the configurations for your Hadoop/Hive instance.

Step 4: Configure the Campaign Environment Variables

Navigate to the Campaign Home directory and then open the setenv.sh file in the /bin directory. Update the LD_LIBRARY_PATH environment variable to include the /lib/64 path from your Hive ODBC driver directory. You must also add a new variable named ODBCINI, which points to the odbc.ini file you edited in the previous step.

Step 5: Importing the Hive Data Source Template

IBM Campaign versions 9.1.2 and later will include a Hive data source template which can be imported into the configurations. Locate the file named BigDataODBCHiveTemplate.xml within the /conf directory of Campaign Home.

HiveDataSource 1

Navigate to the Platform Home, then to the /tools/bin directory. Inside this directory, you will find configTool.sh, which is used to edit Platform’s configuration settings.

Use the command:

./configTool.sh -i -p “Affinium|Campaign|partitions|partition1|dataSources” –f /mnt/ebs1/ibm/EMM/Campaign/conf/BigDataODBCHiveTemplate.xml

HiveDataSource 2
Step 6: Configuring the Hive Data Source

Within the Unica application, navigate to the Platform configurations via Settings -> Configuration. Locate the dataSources configurations in Campaign|partitions|partition1|dataSources. When you expand dataSources, you should see the option for (BigDataODBCHiveTemplate). If this option isn’t present, then the configTool command on the previous step was unsuccessful.

Enter all required properties into the template for your Hadoop/Hive instance.

**Note: Ensure the DSN property matches the name specified in the odbc.ini file.

ConfiguringHive 3
Step 7: Adding Test Data

To test your Hive integration, you’ll need to create a test table and insert some data to query. For this example, we will use a file which represents data that was taken from an online mortgage tool and transformed to a simple .csv file.

Place your test .csv file into the HDFS using the hdsfs dfs command.

AddingTestData 1
AddingTestData 2
AddingTestData 3
Step 8: Creating a Test Table

Next, create the Hive table using this command.

CreatingTestTable 1

You must then ensure that the data types reflect the data that is in our test.csv file.

CREATE EXTERNAL TABLE IF NOT EXISTS
MORTGAGE_ENGAGEMENT_WL(
CookieID INT,
time STRING,
amortPeriod INT,
term INT,
rate INT,
housePrice INT,
downpayment INT,
payFreq STRING)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ‘,’
STORED AS TEXTFILE
location ‘/home/hadoop/test’;

Step 9: Adding Data to Test Table

Next, load the data from the MRTG_TOOL.csv file into your Hive table.

load data local inpath ‘/home/hadoop/MRTG_TOOL.csv’ into table MORTGAGE_ENGAGEMENT_WL;

AddingDataTestTable 1
Step 10: Mapping the Hive Tables

For this step, you’ll need to map your Hive tables. This is accomplished the same way as regular user table mapping. Navigate to Settings -> Campaign Settings -> Manage table mappings.

Since Hive tables will always be User Tables as opposed to System Tables, click on Show User Tables then click New Table.

Select Base record table, then select your Hive data source that you created with the template in the Platform Configurations.

Select one of your Hive tables from the Source table drop menu. The Hive table fields should automatically populate. You can then change the name of the table that will appear in Campaign. Ensure your field names match as expected, then click Next and enter the name for the table within Campaign.
Select one of your Hive tables from the Source table drop menu. The Hive table fields should automatically populate. You can then change the name of the table that will appear in Campaign. Ensure your field names match as expected, then click Next and enter the name for the table within Campaign.
Step 11: Confirm the Table Mapping

Finally, you will test to ensure that you can access the data in our table. Open any flowchart and select the new table as the Input in a process box.

Profile any field in the table to confirm that the data is present and can be queried.

Still With Us?

If you are an experienced Unica Campaign administrator, you most likely noticed the commonalities with configuring traditional relational database sources – UDI (Universal Dynamic Interconnect), ODBC and flowchart process boxes remain relatively unchanged. Even so, those similarities are only skin deep, as working with Hadoop data sources typically demands a rethinking of how your campaigns can wield previously untouched, unstructured data.

If you are still struggling with your integration after reading the above steps, please reach out for advice and suggestions.

nick ford

Nicholas Ford

MUNVO CONSULTANT

Nick is passionate about looking under the hood of complex marketing automation systems and explaining them to a diverse audience. This interest has lead Nick to deliver interactive and personalized experiences for clients in the telco, financial and retail industries. Nick is motivated by helping his clients to adopt and integrate emerging technologies such as Hadoop and cloud into their marketing ecosystems.

Talk To Our Experts Today

All fields marked * are required

Sales Inquiries + 1 (514) 223 3648
General Inquiries + 1 (514) 392 9822
sales@munvo.com

© 2024 Munvo is a trademark of Munvo Solutions Inc.


Pin It on Pinterest

Plus Company Privacy Policy

Underline

(Canada) EN

Updated as of September 11th. 2023

“PLUS COMPANY” AND “COMPANY” MEAN PLUS COMPANY CANADA INC. AND ITS AFFILIATES AND BUSINESS UNITS IN CANADA.

Plus Company respects the privacy of its customers.

This Policy concerns you. It describes how we collect, use, disclose and protect your personal information, including when you visit our website or any website we own, operate or control (collectively, the “Site”), when you contact us by phone or email or when you communicate with us via social media.

We may update this Policy (see “Changes to the Policy” below).

You should read this entire Policy before submitting information to us or using our Site. If you submit personal information to us, we assume that you authorize us to use and disclose it as described in this Policy.

What is personal information?

Personal information is information that identifies you directly or indirectly, on its own or with other information, such as your name, contact details or IP address.

We may make full use of all information that is de-identified, aggregated or otherwise not in personally identifiable form.

Information collection and use

We collect personal information …

  • From you
  • Using automated technology when you visit our Site or communicate with us through email
  • When third parties, such as service providers, collect such personal information on our behalf

When do we collect your personal information?

  • When you browse our site
  • When you contact us by phone or email
  • When you communicate with us via social media

What type of personal information do we collect?

  • Usage information – IP address, information on the device, domain name, browser used to visit our site, pages displayed, time spent on a page, link clicks
  • With your consent, we may have access to your geolocation and
    site-navigation patterns.
  • Information you provide to us, such as personally identifiable information or contact information, as well as your comments or questions.

Why do we need it?

  • To better understand how you use our Site.
  • To modify or improve our Site, services and business activities.
  • For marketing, research and legal purposes.
  • To comply with our policies, procedures and legal obligations, including complying with law enforcement or governmental authority requests, investigating fraudulent activity, resolving disputes and enforcing our legal agreements and policies.

With whom do we share this personal information?

As part of our business operations, we may disclose personal information to the following categories of third parties:

  • Consultants, service providers and contractors we use to support our business and operations (e.g., hosting or operating our Site and services, data collection, reporting, Site metrics and analytics, data analysis, fraud detection services) who have agreed to keep the information confidential and use it only to provide the applicable services.
  • Government authorities, agencies, if required to do so by law, regulation or court order, or to respond to governmental and/or law enforcement requests.
  • An acquirer or successor in interest in the event of a reorganization, merger, sale, change of control, consolidation, joint venture, assignment, transfer or other disposition of all or any part of Plus Company or its affiliates, including any negotiation thereof.

Where is your personal information stored?

We currently retain personal information in North America.

We may disclose personal information in locations other than your country, province or state of residence, where privacy laws may differ.

If your personal information is used outside your country, province or state of residence, it is subject to the laws of the place where it is located and may be disclosed to governments, courts, law enforcement agencies or regulatory bodies of that place, or disclosed in accordance with the laws of that place. However, our practices regarding your personal information will remain governed by this Policy and by applicable privacy laws.

Retention of personal information

We will retain your personal information (collected through online and offline methods) for as long as it is necessary for the purposes described in this Policy. We will also retain and use your personal information to the extent necessary to comply with our legal obligations, resolve disputes and enforce our legal agreements and policies.

Intended audience of Site

Our Site, social media accounts and online activities are intended for persons aged 17 and over. Therefore, we do not request personal information from minors aged 16 years or younger, nor do we knowingly collect such information from minors aged 16 or younger. If you are not 17 or older, you should not visit or use our Site.

How do we protect your personal information?

We take reasonable, appropriate steps to protect personal data from loss, misuse and unauthorized access, disclosure, alteration or destruction, whether in transmission or storage. Remember, however, that no security system is infallible and that transmission over the Internet is not perfectly secure or error-free.

We use a secure server. Only authorized persons have the right to access this information, which they are required to keep strictly confidential.

Your rights with regard to your personal information

Right to access and correct

You may request access to and obtain a copy of the personal information we hold about you.

If any personal information about you is inaccurate, incomplete or ambiguous, or if the collection, disclosure or retention of such personal information is not permitted by law, you may require that it be rectified.

You can also ask us for information about the source of your personal information (if it was not obtained from you), as well as the names of persons who have access to your information and details about how long it is kept.

Right to withdraw your consent

You may request to withdraw your consent to our use or disclosure of your personal information.

In most cases, withdrawing your consent means that we will no longer be able to offer you certain services. Otherwise, we will inform you of the consequences of refusal in our request for consent.

To exercise your rights, please send a request in writing, along with proof of identity, to our Privacy Officer at the contact information provided under How to contact us.

Once your request has been received, we will respond in writing within 30 days.

Cookies and other automated technology

A “cookie” is a small text file that is placed onto an Internet user’s web browser or device and is used to remember and/or obtain information about the user. A “web beacon” is a small object or image that is embedded into a web page, application or email and is used to track activity. Web beacons are also sometimes referred to as pixels and tags. We and/or third parties, including our service providers on our behalf, may use cookies, web beacons and other similar technology to collect information for the purposes described in this Policy, including analytics and monitoring performance and improvement of our Site (traffic, errors, page load time, popular pages, etc.). Before using these technologies to geolocate you or evaluate certain characteristics about you, such as your personal preferences, interests or online behaviour, we will ask for your permission by informing you how to enable such features. We use Google Analytics to understand how our Site, services and products perform and how you use them. To learn more about how Google processes your data, visit https://policies.google.com/privacy. To change your settings relating to cookies and other automated technology, visit our Cookie Manager.
For information and questions about the use of your personal information or this Policy, you may contact our Privacy Officer at privacy@pluscompany.com.
This Policy may be revised from time to time for any reason. In such a case, we will notify you of such changes by indicating the date of the last update. Review the Policy before submitting personal information or using our Site, services or social media pages.
Search