1 / 14

Daily Batch Process for Demand Forecasting

Daily Batch Process for Demand Forecasting. 2019-01-16. Uploads and Imports. Prepare a roadmap of your batch processes – files, queries, import actions / processes in Anaplan. Consistent Naming. Script name BAT-01a Product Hierarchy.bat

jana
Download Presentation

Daily Batch Process for Demand Forecasting

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Daily Batch Process for Demand Forecasting 2019-01-16

  2. Uploads and Imports Prepare a roadmap of your batch processes – files, queries, import actions / processes in Anaplan

  3. Consistent Naming • Script name BAT-01a Product Hierarchy.bat • Anaplan Action (Process) name BAT-01a Product Hierarchy • Reject log name BAT-01a Product Hierarchy.log • AC Process log nameBAT-01a Product Hierarchy 2019-01-16 0800.log

  4. Don’t Rename Data Hub Processes! • Use Anaplan Processes as wrappers for import actions. • Import job scripts have the name of the Anaplan Process hardcoded in them. • It is okay to rename Import Actions. Just don’t change Process names.

  5. Other Tasks • Pre-import processing Test for file creation process completion, notify Anaplan admin if upstream job was delayed Copy files to an upload folder. Avoids a potential conflict when file processing is delayed. Set current date • Post-import processing Cancelled order purge Delete files from upload folder Email notification of process completion Kill hanging java.exe sessions after 30 minutes (Short-term workaround for AC 1.4 bug)

  6. Anaplan Connect versions (as of Jan 2019) • 1.3.3.5 – Most stable, but missing 2 key features. No CA Certificate authentication No jdbc.properties • 1.4 – Hangs in one specific type of reject Bug: cmd and java.exe threads left hanging on specific duplicate record reject Workaround: Kill scheduled jobs after 30 minutes and kill all java.exe after batch is done. • 1.4.1 Due out Feb 2019. Should be best of all worlds. Okay to have some of each: 1.3.3.5 for BI File uploads and 1.4 for SQL queries Recommend consolidating to 1.4.1 when released.

  7. Anaplan Service Account A service account like svcanaplan@customer.com is used to login to Anaplan in all scheduled batch jobs. This account has an Outlook email box … necessary for setting up the account in Anaplan and for password resets. Anaplan Connect does NOT support SSO. This account’s Anaplan password is maintained in Anaplan and will expire every 90 days. SET AN OUTLOOK REMINDER TO UPDATE PASSWORD APRIL 1 AND REPEATING AT 90 DAY INTERVALS. Store info like login credentials and connect strings for jdbc in a GlobalVariables.bat file on Windows server … single place to update when it expires. Future state: CA Certificate authentication with certificate that expires once a year. More secure, less maintenance. Requires AC 1.4+ only (wait for next release due in Feb).

  8. Anaplan Connect Scripts • Set variables unique to this job. File, SQL query, Action, Log Name • Get connection info from Globals.bat • If SQL, run CreateJDBC.Properties.bat • Assemble the OPERATION= line Connection, upload, action, and reject log flags • Run AnaplanClient.bat, record the chatter to process log file • Check ERRORLEVEL and log for errors, send email if trouble • Exit

  9. CreateJDBCProperties.bat • Custom script collects connection, credentials, and SQL query information from other files and assembles a short-lived plaintext file for Anaplan Connect’s needs. • Globals.bat contains connection and credentials. Intention is to encrypt that file to avoid plaintext password storage. • SQLQueries.txt contains the SQL statements* used by jobs: ::QueryName Select ABC, DEF From Table Where GHI<0 ::end * No double quotes allowed in SQLQueries.txt, other special characters (<>=&|) are ok.

  10. CreateJDBCProperties info online A best practices forum post in Community.Anaplan.com contains full details.https://community.anaplan.com/t5/Best-Practices/Windows-cmd-shell-script-for-dynamically-generating-JDBC

  11. Logging • Import action rejects (if any) are saved to file in\\<servername>\Anaplan\ErrorLogs\ • Process “chatter” while AnaplanClient is interacting with Oracle and Anaplan is captured on the Windows server atD:\Anaplan\ProcessLogs\Process logs are named with <job name> <date> <start time>.log • History in Anaplan

  12. Rejected Record Logs • Same information as found on Details tab after a manual import into Anaplan. • No rejects or warnings = no file. • Reject files are overwritten daily. No retention. • Could add Date Time to filename and retain for 30 days, same as Process Logs.

  13. Process Logs • Connection • SQL query • Upload • Import – summary stats • To List • To Module Process logs are retained for 30 days to allow some forensics on past trouble or performance changes.

  14. Scheduling • Windows Task Scheduler on Windows server Simple scheduler, limited features. Most data centers have a more robust solution that monitors errors and notifies support team, branches on conditions, handles job dependencies… Setting jobs up can be a significant project. • Two primary jobs start at 8 am File import jobs (AC 1.3.3.5) SQL jobs (AC 1.4) These could be merged after AC 1.4.1 is released with bug fix. • Cleanup task at 8:30 to delete copies of data files and kill java.exe threads (bug workaround)

More Related