How can I apply quotes to a URL in a .bat script?

Question

The general form of a curl command in CMD that works is:

curl -o latest.dump -L "https://s3.amazonaws.com/hkpgbackups/[email protected]/xxxx.dump?AWSAccessKeyId=AKIAJSCBEZJRDOTGNGZQ&Expires=1411489500&Signature=%2F5zwQNZNN0H2XSR4wSqQ%2FFExBdI%3D"

But the url is dynamically generated so for me to put this into a .bat script. OK, no problem so here's my .bat script that on paper should work.

@echo off

for /f "delims=" %%a in ('heroku pgbackups:url') do @set this_url=%%a
ECHO %this_url%
curl -v -o latest.dump -L %this_url%

But the output bails because at the first '&' as you might expect. The echo of this_url shows the full string with quotes. How can I append, or force, to encapsulate this string with quotes?

I have attempted but it returns the same type of error:

cmd /k ""curl -v -o latest.dump -l" "('heroku pgbackups:url')""

Some may ask why not do this in Unix, or CYGWIN? I did that first. The command works perfectly on my OS X personal machine. So it should work in CYGWIN? Well I get the same type of errors.

I've attempted incorporating a Perl script using the uri_escape library and inserted it. But the URL it passes is not recognized by curl.

Forcing the output within single quotes munges the URL in that it starts correct but the closing single quote is placed within the last few characters of the URL which renders it useless to curl.

The output of heroku pgbackups:url in the CMD console is:

"https://s3.amazonaws.com/hkpgbackups/[email protected]/b080.dump?AWSAccess
KeyId=AKIAJSCBEZJRDOTGNGZQ&Expires=1411657655&Signature=MqzqBitAN7MADiLAsdAN4ZQk
RH0%3D"

The output of the .bat file above is:

https://s3.amazonaws.com/hkpgbackups/[email protected]/b080.dump?AWSAccessK
eyId=AKIAJSCBEZJRDOTGNGZQ
'Expires' is not recognized as an internal or external command,
operable program or batch file.
'Signature' is not recognized as an internal or external command,
operable program or batch file.
* About to connect() to s3.amazonaws.com port 443 (#0)
*   Trying 205.251.243.66...   % Total    % Received % Xferd  Average Speed   Ti
me    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0co
nnected
* Connected to s3.amazonaws.com (205.251.243.66) port 443 (#0)
* SSLv3, TLS handshake, Client hello (1):
} [data not shown]
* SSLv3, TLS handshake, Server hello (2):
{ [data not shown]
* SSLv3, TLS handshake, CERT (11):
{ [data not shown]
* SSLv3, TLS handshake, Server finished (14):
{ [data not shown]
* SSLv3, TLS handshake, Client key exchange (16):
} [data not shown]
* SSLv3, TLS change cipher, Client hello (1):
} [data not shown]
* SSLv3, TLS handshake, Finished (20):
} [data not shown]
* SSLv3, TLS change cipher, Client hello (1):
{ [data not shown]
* SSLv3, TLS handshake, Finished (20):
{ [data not shown]
* SSL connection using AES128-SHA
* Server certificate:
*        subject: C=US; ST=Washington; L=Seattle; O=Amazon.com Inc.; CN=s3.amazo
naws.com
*        start date: 2014-04-12 00:00:00 GMT
*        expire date: 2015-04-13 23:59:59 GMT
*        subjectAltName: s3.amazonaws.com matched
*        issuer: C=US; O=VeriSign, Inc.; OU=VeriSign Trust Network; OU=Terms of
use at https://www.verisign.com/rpa (c)10; CN=VeriSign Class 3 Secure Server CA
- G3
*        SSL certificate verify result: unable to get local issuer certificate (
20), continuing anyway.
> GET /hkpgbackups/[email protected]/b080.dump?AWSAccessKeyId=AKIAJSCBEZJRD
OTGNGZQ HTTP/1.1
> User-Agent: curl/7.21.7 (amd64-pc-win32) libcurl/7.21.7 OpenSSL/0.9.8r zlib/1.
2.5
> Host: s3.amazonaws.com
> Accept: */*
>
< HTTP/1.1 403 Forbidden
< x-amz-request-id: 4566C5AB208C1248
< x-amz-id-2: zF/zr+fbR/pE7nvF7vvUmOdZQeMzjSBI6SPLKH14LGAI5JAb2xyoLhuuDGBDhqcq
< Content-Type: application/xml
< Transfer-Encoding: chunked
< Date: Thu, 25 Sep 2014 14:51:46 GMT
< Server: AmazonS3
<
{ [data not shown]
100   231    0   231    0     0    592      0 --:--:-- --:--:-- --:--:--   704
* Connection #0 to host s3.amazonaws.com left intact
* Closing connection #0
* SSLv3, TLS alert, Client hello (1):
} [data not shown]
'Expires' is not recognized as an internal or external command,
operable program or batch file.
'Signature' is not recognized as an internal or external command,
operable program or batch file.

The 403 error indicates that the url got munged, but otherwise it passed through or I would have gotten a 400 error. For some reason in the script, the "&" get redirected because they're interpreted in the script.

If you were me, how would you run this command dynamically pass this URL so it's not interpreted by CMD, bash, etc? thanx, sam

Answer 1

Looking at your originally posted script:

@echo off

for /f "delims=" %%a in ('heroku pgbackups:url') do @set this_url=%%a
ECHO %this_url%
curl -v -o latest.dump -L %this_url%

If the output of ECHO %this_url% is exactly as you posted, with quotes around it, then your existing batch script should work.

If the value of %this_url% does not have quotes, then you should add quotes to the curl command:

curl -v -o latest.dump -L "%this_url%"

You could modify the command to make it work regardless whether %this_url% contains quotes by removing any existing quotes, and explicitly adding your own:

curl -v -o latest.dump -L "%this_url:"=%"

The other command you list looks completely wrong to me:

cmd /k ""curl -v -o latest.dump -l" "('heroku pgbackups:url')""

The outer quotes should not be needed, nor should the quotes around "curl -v -o latest.dump -l". You could put quotes around the exe file only "curl", but that should not be needed unless you include the path to curl and the path includes spaces or poison characters.

I simply do not understand the "('heroku pgbackups:url')" syntax at all.