Disclaimer: This article is meant for a technical audience
The purpose of this article will to provide introduction to hosting requirements for EMAE Service, used as middleware for CloudVirga iMP communication with Ellie Mae Encompass LOS. This article will cover the following topics
- EMAE Service
- EMAE Service Core
- Hardware/Software Requirements for Hosting EMAE Service
- Network Requirements
- Authorization Server
- SQL Server
- High Availability
- Package Content
- Deployment Instructions
- Service Deployment
- Database Deployment
EMAE Service consists of components depicted on diagram below:
- EMAE Service – Core component of EMAE service. This is a component that is accepting requests from iMP and communicating with Encompass instance to process those requests.
- EMAE Configuration Database – Database containing configurations for EMAE service, including field mapping instructions
- EMAE Logging DB – Database containing logging and auditing data for EMAE service. This is transient data used primarily for troubleshooting, statistical and performance analysis and is usually transient.
- Authorization Server – Optional component used in combination with REST interface for EMAE Service. It can be replaced with other OWIN OAuth 2.0 compatible authorization server.
- Authorization Server DB – Optional database used for default implementation of Authorization Server
EMAE Service Core
EMAE Service core component is a standalone Windows Service.
Hardware/Software Requirements For hosting EMAE Service
Hosting requirements are at a minimum Windows 2012R2 with .NET framework 4.5.
CPU and memory requirements depend on projected volume, type of interactions with Encompass and scaling choice (horizontal vs. vertical).
Minimal recommendation is 2 CPU cores and 8 GB RAM, equivalent to AWS m5.large EC2 instance or Azure Standard_D2_v3 VM instance.
Hosts for EMAE Service need to have Encompass SDK installed, matching the version of target Encompass environment.
EMAE service has two possible interfaces it exposes to clients. One or both interfaces must be available to iMP to communicate with EMAE service.
Rest interface is standard HTTPS REST endpoint exposed to consumers by EMAE Service. It is a recommended way of exposing EMAE Service to iMP.
It is typically hosted on standard HTTPS TCP 443, but TCP port can be changed as needed. This implementation relies on usage of OWIN OAuth 2.0 Authorization Server in combination with HTTPS transport layer security. This implementation can optionally be turned off in case security is handled by external components/layers in front of EMAE service.
WCF interface consists of two WCF services with separate base URLs. For clients internal to EMAE Service (intranet clients) this is typically set up as a pair of standard NetTcpBinding bindings on tcp ports 60052 and 60054. These are just reference ports and can be changed as needed.
For external clients (internet scenario) these bindings are typically configured as WsHttpBinding with standard Transport security combined with basic or client certificate authentication.
EMAE Service can be scaled horizontally using multiple instances of service behind standard load balancers sharing single backend SQL server and configuration.
Depending on types of interfaces being exposed (REST or WCF, NetTcpBinding or WsHttpBinding) and type of load balancer used, configurations of load balancers may slightly vary so these are handled on case by case basis.
It is recommended to host several instances of EMAE Service to ensure High Availability as well as enable rolling upgrades of service with no outage. Service is implemented in such way that it supports rolling deployment or blue-green deployment scenarios.
Default implementation of Authorization server is hosted within EMAE service itself. Service can be configured to use compatible external authorization server.
Additionally, usage of Authorization server by EMAE service can be disabled using configuration. This is useful mostly in cases where it is deemed not required to perform authorization of service calls themselves. This is mostly the case for internal hosting where strict security is implemented on infrastructure level, and/or for hosting where service is exposed through third party solutions such as API management solutions that handle security for service calls.
More details on implementation of external Authorization Server will be provided in separate document.
Databases used for EMAE Service – Configuration, Logging and Authorization are Microsoft SQL Server databases. Recommended version of MSSSQL is SQL Server 2016 Standard edition.
Setting up MSSQL in one of standard High Availability models is recommended for production to ensure service availability.
For more details please consult Microsoft SQL Server documentation on High Availability solutions, for example:
EMAE Service package consists of:
- Windows installer MSI file for installation of EMAE Service
- Database installer
- Windows installer MSI file that installs prerequisites for automated deployment of database packages (standard Data-tier Application Component Packages or dacpacs). This file is sufficient to install required databases on SQL server
- Zip file containing Data-tier Application Component Packages in case of manual deployment of databases is preferred
Any one of two options for installing databases is sufficient.
- Deployment instructions document and deployment scripts
This section of document outlines basic steps for deployment of EMAE service to individual server.
EMAE Service MSI is intended for silent deployment using set of parameters provided in command line. For convenience, PowerShell script with script configuration file will be provided in package along with instructions for setting up parameter files per environment.
Database deployment can be performed either using database MSI, or manually by deploying Data-tier Application Component Packages to SQL server. Database MSI is deployed in similar way to EMAE Service MSI, using provided PowerShell script and appropriate configuration file for silent install.
Manual deployment can be done using one of several standard methods, from command line, PowerShell, SQL Server Management Studio etc. Documentation for these methods is provided by Microsoft and available online.