ExternalResourceClient

Overview

In Sugar 12.1, ExternalResourceClient was introduced to replace cURL. This has also been backported to all supported versions of Sugar. Therefore any apps or integrations for Sugar that use a Module Loadable Package (MLP) that includes any of the PHP curl_*, socket_*, or stream_* functions will need to change to use the ExternalResourceClient lib.

Self-Signed Certificates

ExternalResourceClient is designed to be as secure as possible. We will not support disabling the check for authenticity of the peer's certificate as you could in cURL with using the CURLOPT_SSL_VERIFYPEER option. This is not a security best practice and is therefore not allowed in the new client.

If your target URL has an invalid or self-signed certificate, you can benefit from using Let's Encrypt. Let's Encrypt is a free, automated, and open Certificate Authority (CA) that makes it possible to set up an HTTPS server and have it automatically obtain a browser-trusted certificate without any human intervention. You can get started here.

TLS Mutual Authentication

ExternalResourceClient supports TLS mutual authentication where it ensures that the parties at each end of a network connection are who they claim to be by verifying that they both have the correct private key. The information within their respective TLS certificates provides additional verification is designed to be as secure as possible. In cURL it could be achieved using the curl_setopt($ch, CURLOPT_SSLCERT, <CERT_FILE>); and curl_setopt($ch, CURLOPT_SSLKEY, <KEY_FILE>); options.

ExternalResourceClient can be configured with the following options or any of those available in the PHP SSL Context options.:

local_cert - Path to local certificate file on filesystem. It must be a PEM encoded file which contains your certificate and private key. It can optionally contain the certificate chain of issuers. The private key also may be contained in a separate file specified by local_pk.

local_pk - Path to local private key file on filesystem in case of separate files for certificate (local_cert) and private key.

  <?php
$client = (new ExternalResourceClient())
    ->setSslContextOptions([
        'local_cert' => '<CERT_FILE>', //CURLOPT_SSLCERT analogue
        'local_pk' => '<KEY_FILE>', // CURLOPT_SSLKEY
    ]);

IP vs Domain/Hostnames via DNS

ExternalResourceClient does not support using an IP address in your URL. Editing /etc/hosts on your server is a very easy and fast way of accomplishing this, but if that is not an option and you have a proper DNS server configured, you can use one of the following articles to help you understand what DNS is and how you can configure them in different OSs and infrastructures.

• Build your own DNS server on Linux
• DNS Configuration: Everything You Need to Know
• How to Setup DNS Server on Windows Server 2012

Note: If your domain is behind a LoadBalancer, ExternalResourceClient will reverse DNS it for an IP and that resolution may bring different IPs every time. LoadBalancer will balance the load to the IP at that particular moment and return it to ExternalResourceClient. If you are in a restricted DMZ behind a firewall and need to open connections to those IPs, you must get all IPs served by the LoadBalancer and add them all to your firewall.

DNS Over HTTPS Configuration

The security.use_doh configuration setting enables remote Domain Name System (DNS) resolution via the HTTPS protocol. A goal of the method is to increase user privacy and security by preventing eavesdropping and manipulation of DNS data by man-in-the-middle attacks by using the HTTPS protocol to encrypt the data between the DoH (DNS over HTTPS) client and the DoH-based DNS resolver.

When enabled, it will use dns.google (8.8.4.4) to resolve hostnames.

For more details on the security.use_doh configuration setting, see the Core Settings page.

Local Endpoints

If you do not know what server-side request forgery (SSRF) is, please refer to this detailed explanation and examples of how an attacker can reach your local network using malicious HTTP requests.

ExternalResourceClient prevents this type of attack by taking the resolved IP from your URL and checking against a Sugar Config $sugar_config['security']['private_ips'] array list.

This list contains a wide range of internal IPs blocked by default [ '10.0.0.0|10.255.255.255', '172.16.0.0|172.31.255.255', '192.168.0.0|192.168.255.255', '169.254.0.0|169.254.255.255', '127.0.0.0|127.255.255.255'] .

For example, if your URL is http://an-internal-url/api/get and "an-internal-url" resolves to an IP 10.0.0.87, it will throw an exception because it is within the range of your config.

Add your critical IP ranges to this config so no attacker will get to sensitive IPs through ExternalResourceClient's HTTP Request.

Sugar Proxy

To get proxy settings, ExternalResourceClient relies on Administration::getSettings('proxy'). This utility queries Sugar's infrastructure (cache or DB) to retrieve those settings, therefore it needs to be bootstrapped into SugarCRM.

In the case of MLPs that use standard modules via the web, you don't need to manually bootstrap SugarCRM, so you may use ExternalResourceClient in your methods without requiring entryPoint.php

If you are writing a CLI script, you must include SugarCRM entryPoint.php:

  <?php
if (!defined('sugarEntry')) {
   define('sugarEntry', true);
}

define('ENTRY_POINT_TYPE', 'api');
require_once 'include/entryPoint.php';

?>

Test your customization

You can easily test your package by enabling PackageScanner checks in the supported versions for now ($sugar_config['moduleInstaller']['enableEnhancedModuleChecks'] = true), remember, it comes disabled by default in 12.1.

If your code doesn't go through PackageScanner, it will not be enforced, that's the premise we're working with.

Examples of Replacing cURL

The below examples can be used as a guideline for replacing your current cURL code with ExternalResourceClient.

You must import this client and its exceptions (if needed) from:

  <?php

use Sugarcrm\Sugarcrm\Security\HttpClient\ExternalResourceClient;
use Sugarcrm\Sugarcrm\Security\HttpClient\RequestException;

HTTP GET

The following code snippet provides you with a code replacement from cURL GET to ExternalResourceClient.

Before:

  // cURL GET
$ch = curl_init();

$url = 'https://httpbin.org/get';
curl_setopt_array($ch, array(
    CURLOPT_URL => $url,
    CURLOPT_RETURNTRANSFER => true,
    CURLOPT_ENCODING => "",
    CURLOPT_MAXREDIRS => 10,
    CURLOPT_TIMEOUT => 60,
    CURLOPT_FOLLOWLOCATION => true,
    CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_2TLS,
));

$response = curl_exec($ch);

$httpCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
$errorNo = curl_errno($ch);
$error = curl_error($ch);

if ($errorNo !== 0) {
    $GLOBALS['log']->log("fatal", "curl error ($errorNo): $error");
    throw new \SugarApiExceptionError("curl error ($errorNo): $error");
}

curl_close($ch);

if ($httpCode === 0 || $httpCode >= 400) {
    $GLOBALS['log']->log("fatal", "Error message: $error");
    throw new \SugarApiException($error, null, null, $httpCode ? $httpCode : 500);
}

echo $response;

After:

  // ExternalResourceClient GET
try {
    // Set timeout to 60 seconds and 10 max redirects
    $response = (new ExternalResourceClient(60, 10))->get($url);
} catch (RequestException $e) {
    $GLOBALS['log']->log('fatal', 'Error: ' . $e->getMessage());
    throw new \SugarApiExceptionError($e->getMessage());
}
$httpCode = $response->getStatusCode();
if ($httpCode >= 400) {
    $GLOBALS['log']->log("fatal", "Request failed with status: " . $httpCode);
    throw new \SugarApiException("Request failed with status: " . $httpCode, null, null, $httpCode);
}
echo $response->getBody()->getContents();

Note that timeout cannot be 0 or negative.

HTTP POST

The following code snippet provides you with a code replacement from cURL POST to ExternalResourceClient.

Before:

  // cURL POST JSON
$url = 'https://httpbin.org/post';

$ch = curl_init($url);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 1);
curl_setopt($ch, CURLOPT_HTTPHEADER, array ("Content-Type: application/json"));
curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode(['foo' => 'bar']));

$response = curl_exec($ch);
curl_close($ch);
$parsed = !empty($response) ? json_decode($response, true) : null;
var_dump($parsed);

After:

  // ExternalResourceClient POST JSON
try {
    $response = (new ExternalResourceClient())->post($url, json_encode(['foo' => 'bar']), ['Content-Type' => "application/json"]);
} catch (RequestException $e) {
    throw new \SugarApiExceptionError($e->getMessage());
}

$parsed = !empty($response) ? json_decode($response->getBody()->getContents(), true) : null;
var_dump($parsed);

Authenticated Endpoints (CURLOPT_USERPWD)

The following code snippet provides you with a code replacement from cURL to ExternalResourceClient passing authentication params.

Before:

  $username = 'foo';
$password = 'bar';
$url = 'https://httpbin.org/basic-auth/{$username}/{$password}';

$ch = curl_init($url);
curl_setopt($ch, CURLOPT_USERPWD, $username . $password);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode(['foo' => 'bar']));

$response = curl_exec($ch);
curl_close($ch);
$parsed = !empty($response) ? json_decode($response, true) : null;
var_dump($parsed);

After:

  $username = 'foo';
$password = 'bar';
$auth     = base64_encode( "{$username}:{$password}" );
 //Passing custom 'Authorization' header
// try + catch is omitted for brevity
$response = (new ExternalResourceClient())->get("https://httpbin.org/basic-auth/{$username}/{$password}", [
    'Authorization' => 'Basic ' . $auth,
]);
echo $response->getBody()->getContents();

// OR the same by using user:password@hostname.tld URL format
 // try + catch is omitted for brevity
$response = (new ExternalResourceClient())->get("https://{$username}:{$password}@httpbin.org/basic-auth/{$username}/{$password}");
echo $response->getBody()->getContents();

Stream

The following code snippet provides you with a code replacement from stream to ExternalResourceClient.

Before:

  <?php
// Sending GET
if ($stream = fopen('https://httpbin.org/get', 'r')) {
    echo stream_get_contents($stream);
}

// Sending POST
$options = [
    'http' => [
        'method' => 'POST',
        'header' => ["Content-Type: application/x-www-form-urlencoded"],
        'content' => http_build_query(['fieldName' => 'value', 'fieldName2' => 'another value']),
        'ignore_errors' => true,
    ],
];
$context = stream_context_create($options);
if ($stream = fopen('https://httpbin.org/post', 'r', false, $context)) {
    echo stream_get_contents($stream);
}

After:

  // Using ExtenalResourceClient
// GET
echo (new ExternalResourceClient())
->get('https://httpbin.org/get')
->getBody()
->getContents();
// POST
echo (new ExternalResourceClient())
->post(
'https://httpbin.org/post', 
['fieldName' => 'value', 'fieldName2' => 'another value']
)
->getBody()
->getContents();

Socket

The following code snippet provides you with a code replacement from socket to ExternalResourceClient.

Before:

  <?php 

// Sending GET
$fp = fsockopen("httpbin.org", 80, $errno, $errstr, 30);
if (!$fp) {
    echo "$errstr ($errno)\n";
} else {
    $out = "GET /get HTTP/1.1\r\n";
    $out .= "Host: httpbin.org\r\n";
    $out .= "Connection: Close\r\n\r\n";
    fwrite($fp, $out);
    while (!feof($fp)) {
        echo fgets($fp, 128);
    }
    fclose($fp);
}
// Sending POST

$fp = fsockopen("httpbin.org", 80, $errno, $errstr, 30);
if (!$fp) {
    echo "$errstr ($errno)\n";
} else {
    $postData = http_build_query(['fieldName' => 'value', 'fieldName2' => 'another value']);
    $out = "POST /post HTTP/1.1\r\n";
    $out .= "Host: httpbin.org\r\n";
    $out .= "Content-type: application/x-www-form-urlencoded\r\n";
    $out .= "Content-length: " . strlen($postData) . "\r\n";
    $out .= "Connection: Close\r\n\r\n";
    $out .= "{$postData}\r\n\r\n";
    fwrite($fp, $out);
    while (!feof($fp)) {
        echo fgets($fp, 128);
    }
    fclose($fp);
}

After:

  // Using ExtenalResourceClient
// GET
echo (new ExternalResourceClient())
->get('http://httpbin.org/get')
->getBody()
->getContents();
// POST
echo (new ExternalResourceClient())
->post(
'http://httpbin.org/post', 
['fieldName' => 'value', 'fieldName2' => 'another value']
)
->getBody()
->getContents();

Upload a file

The following code snippet provides you with a code example using ExternalResourceClient to upload files from Sugar.

  // Uploading files from `upload` directory
// File data will be accessible on the target server as $_FILES['file_field_name']
echo 'File upload', PHP_EOL, PHP_EOL;
$file = new UploadFile('file_field_name');
// relative to `upload` directory
$filename = 'f68/2cad6f68-e016-11ec-9c9c-0242ac140007';
$file->temp_file_location = $file->get_upload_path($filename);
// try + catch is omitted for brevity
echo (new ExternalResourceClient())->upload(
    $file,
    'https://httpbin.org/post',
    'POST',
    ['foo' => 'bar'], // Optional, additional form data
    ['custom-header-name' => 'custom-header-value'] // Optional, additional headers
)->getBody()->getContents();