Monocle Gateway throws Proxy error at startup on Win2012R2 Server [SOLVED]

Monocle

@spicymikey said in Monocle Gateway throws Proxy error at startup on Win2012R2 Server:

As far as the port issue. it’s a bit strange. There are no software firewalls on this server. No virus software either. There is a Cisco ASA firewall on the edge of the network but 443 and 80 are explicitly wide open for all traffic (as you would expect). In front of that is AT&T premise equipment to access their Internet service, but no firewall services on there.

The Internet gateway and main network router should not matter as long as the Alexa devices on your network have internal access to port 443 on the server running Monocle Gateway.

I’m curious about your comment regarding “some other program” using the port. This is a server for our facility. Lots of programs on it that use 443. Every time we open a browser and go to an HTTPS site we use that port. I suspect you know that so I might be misunderstanding your suggestion. Are you saying that Monocle needs some kind of exclusive access to 443? if so, then this solutoin probably wont work for us.

If multiple web applications are sharing port 443 on this single server, then its likely that they are all running behind an Apache, NGINX or similar web server. Monocle Gateway does require exclusive access to port 443 meaning it can be the only application listening on that port. This is because Alexa devices will only communicate to camera stream using the RTSP protocol on port 443 (using SSL/TLS encrypted communications.). We can’t share the port with other applications like NGINX or Apache server and we can’t exist as a proxy behind those web servers because it is not using HTTP/HTTPS protocols but rather the RTSP protocol.

You can certainly run the Monocle Gateway service from any other computer on the network, or a small embedded device like a Raspberry Pi or from a virtualized server or a Docker container. There are a lot of options.

You could also assign a secondary IP address to the server’s network card so that you have a secondary IP on which the Monocle Gateway could used to listen to port 443 and not interfere with the existing Apache/NGINX server. This may require some specialized configuration by creating a Monocle properties file to tell the Monocle Gateway which IP is the correct one to listen on, but its certainly possible.

Thanks, Robert

spicymikey

Hi Robert,

Wow, thank you for spending the time to provide such an excellent response. Very much appreciated, and thanks for clearing up in my mind the 443 limits. I misunderstood your answer and thought you were saying it needed exclusive USE. Yes, of course it must be the only device listening for inbound traffic. I’m with you now. As I mentioned this is a Windows server, so although I’m not running Apache or any other Webserver. I am running Microsoft IIS. Really no choice. It’s part of the OS and removing it will damage this computers ability to function as a domain server. I’ll take your advice and move it off of here. it was the logical choice since it’s an “always on” computer here.

Thanks again for thinking this through with me. If the Echo Show 5 is capable of working with your Gateway, I’m sure I will get it to work now with your advice.

Monocle

@spicymikey

Yes, IIS is the Microsoft equivalent of Apache Web Server or NGINX – It’s been a while since I have been in the Microsoft world, so it did not immediately come to mind, but yes this would be occupying port 443 by default — thus the issue we are seeing. Yes, removing or disabling IIS could impact many things – best to avoid that!

The easiest solution is to move it to another computer – but keep in mind a virtual machine or container (Docker) will also work so long as it has it’s own dedicated network interface and IP address on the LAN. Also a tiny Raspberry Pi device (or similar) would also serve well as a dedicated appliance as a long term solution.

Thanks, Robert

spicymikey

Hi Robert

Wanted to give you (and anyone else who reads this thread) an update. It still doesn’t work.

Moved the Monocle Gateway service to another device. That resolved the 443 error.
Changed the Reolink RLC420 stream to one of their substreams. Set it to 640x480. Confirmed the RSTP url worked with VLC and entered it into Monocle.
Even rediscovered the devices from Alexa to be sure.

I’m wondering if anyone has actually gotten it to work with these new Echo Show 5’s. I don’t have an Echo Show to try it with so can’t confirm if it is still something I’m doing wrong, or if these devices won’t work.

Has anyone confirmed the Echo Show 5’s work?

Thanks!

Monocle

@spicymikey

Yes, I have confirmed it working on my Echo Show 5 here in my office using a Reolink camera.

Based on the log, it still looks like the Alexa device is not attempting to access the gateway. Well, actually I don’t see any camera requests in the log file. What is Alexa saying when you request the “Front Yard” camera feed?

Since you moved the gateway to new computer, try generating a new token file. Basically, the old token is tied to the unique DNS hostname that was previously assigned to the old IP address. A DNS update will occur, but sometimes this can take a while. Generating a new token file and using that new token on this new gateway will also generate a new unique DNS hostname.

Thanks, Robert

spicymikey

You saw no activity because the screen grab was before I tried anything. Here’s another shot showing the activity. I also created a new token like you suggested and replaced the old Monocle.token file with the new one. it is being placed in the same folder as the monocle-gateway.exe

Everything seems fine but Alexa just says “Hmmm, the camera does not seem to be responding”.

I tried moving the gateway software out of Program Files (that can sometime cause permission issues with Windows 10. I also ensured the toke file was wide open with security access.

Sounds like you’re not using Windows 10. That might be the issue. Sounds like the only thing different with our environments. I’ll keep trying

Monocle

@spicymikey

Just for clarification, is the IP address detected by Monocle Gateway 10.0.0.6 correct?
Can the Alexa devices on the network access this address?
Is the Monocle Gateway running on physical computer or on a virtual machine?

I’m going to setup a monocle gateway instance in the cloud for you to test with. The issue still seems to be that the Alexa device are not able to open a TCP port to the Monocle Gateway running at 10.0.0.6. This is typically a networking related issue where either a Firewall is blocking access to port 443, the Alexa devices are on a separate network that can’t access the Monocle Gateway such as a protected VLAN, Guest network or the gateway running on a virtual machine without a direct network interface. It can also be an issue of DNS REBINDING. See: https://monoclecam.com/monocle-gateway/troubleshooting/dns-rebinding

Thanks, Robert

Monocle

@spicymikey

OK, go ahead and stop the Monocle Gateway running on your Windows server.

I have setup a gateway server running in the cloud to test with. I have already created a new token file on your Monocle account and configured the cloud gateway server to use this token – so there is nothing you need to do other than stop/disable your local gateway server.

I also have added a new “Demo” camera feed to your Monocle account to test with as well. All you need to do is ask “Alexa, Discover Devices”. She may tell you she discovered a new device named “Demo” or she may say she con’t find any new devices if she already found it automatically before you attempted discovery (she will periodically check for new devices on her own.)

Next, ask “Alexa show me the Demo Camera” and lets see what happens.

If that works, then ask for your “Front Yard” camera.

Thanks, Robert

spicymikey

@Monocle said in Monocle Gateway throws Proxy error at startup on Win2012R2 Server:

@spicymikey

Just for clarification, is the IP address detected by Monocle Gateway 10.0.0.6 correct?
Answer: Yes that is the correct local IP

Can the Alexa devices on the network access this address?
Answer: Yes, all devices on this LAN are getting 10.x IP addresses from a single DHCP server running on the local Windows server

Is the Monocle Gateway running on physical computer or on a virtual machine?
Answer: Both times it was on a physical computer. First on a Dell PowerEdge 2900 Server, now on a Dell Precision 7810 Workstation

I’m going to setup a monocle gateway instance in the cloud for you to test with. The issue still seems to be that the Alexa device are not able to open a TCP port to the Monocle Gateway running at 10.0.0.6. This is typically a networking related issue where either a Firewall is blocking access to port 443, the Alexa devices are on a separate network that can’t access the Monocle Gateway such as a protected VLAN, Guest network or the gateway running on a virtual machine without a direct network interface. It can also be an issue of DNS REBINDING. See: https://monoclecam.com/monocle-gateway/troubleshooting/dns-rebinding
Answer: Great! Thanks for doing this. When I am in the office a little later I will do what you asked and see what happens

Thanks, Robert

Monocle

@spicymikey

PS, I noticed you switched the “Front Yard” camera back to a local IP address from the DNS hostname you were previously using. While using the Monocle Gateway I have setup in the cloud, the local/private IP address won’t work because the cloud gateway cannot access your private network. The “Demo” camera should still work and you can change the “Front Yard” camera back to the DNS hostname if you would like to test it.

I think we are primarily just trying to make sure your entire setup is working with a known good instance of Monocle Gateway (running in the cloud). Once that is vetted, we can concentrate on getting your local copy of Monocle Gateway working properly. With your local copy of Monocle Gateway working, you can then switch the IP cameras back to the private IP addresses.

Thanks, Robert

spicymikey

@Monocle said in Monocle Gateway throws Proxy error at startup on Win2012R2 Server:

@spicymikey

PS, I noticed you switched the “Front Yard” camera back to a local IP address from the DNS hostname you were previously using. While using the Monocle Gateway I have setup in the cloud, the local/private IP address won’t work because the cloud gateway cannot access your private network. The “Demo” camera should still work and you can change the “Front Yard” camera back to the DNS hostname if you would like to test it.

I think we are primarily just trying to make sure your entire setup is working with a known good instance of Monocle Gateway (running in the cloud). Once that is vetted, we can concentrate on getting your local copy of Monocle Gateway working properly. With your local copy of Monocle Gateway working, you can then switch the IP cameras back to the private IP addresses.

Thanks, Robert

Hey Robert,

Good news first. Your Demo camera displays perfectly. So we’ve just eliminated a lot of posible causes.

Regarding the local IP address. Yes I was throwing crap against the wall in the end and trying everything. Thanks for noticing that. I put it back to the public dns name registered on a GoDaddy DNS server.

Bad news is my local camera still doesn’t display the video stream from that camera. Not sure if you need to turn your gateway off first, but I turned mine back on, renamed the rtsp url back to using the public DNS name, and said “Show frontyard camera”. No joy.

You mentioned about this DNS entry you are creating, and i read that white paper you suggested. Your hunch is correct. I tried to ping that FQDN and it failed. Where are you trying to create this DNS entry? If you are putting it on your own public DNS server then I’ll need to add that to my DNS search list. if you are trying to inject it into my local DNS server, well, thats not going to work. You don’t have authority. I guess we have narrowed it down and this DNS entry issue must be the problem! Give me more direction on this

Monocle

@spicymikey
OK, now we are getting somewhere.

I suspect it could be a case of DNS REBINDING. Some routers/Internet Gateways will block public DNS requests when they return a private IP address. Technically there is nothing wrong with this, but it could lead to security risks for Man in the Middle attacks and some networking equipment blocks this by defaults. My PFSense router does.

Or, like you said, perhaps the DNS upstream server is not allowed or there is some issue.

The DNS is publicly registered using Amazon web services. So it should be fully resolvable.

To perform a quick test, you could add this DNS record (6d8e9885-afd1-461a-9857-8e9cbf14bf49.mproxy.io) to your local DNS server and point it to the local IP address (10.0.0.6). That way it will at least resolve for now and if that fixes it we can chase down the real DNS issue.

Thanks, Robert

Monocle

@spicymikey

Here I am using the nslookup utility to validate the DNS hostname.

Here I am testing against my networks default DNS server/resolver:

pi@rpi3bp:~ $ nslookup 6d8e9885-afd1-461a-9857-8e9cbf14bf49.mproxy.io
Server:		10.1.1.1
Address:	10.1.1.1#53

Non-authoritative answer:
Name:	6d8e9885-afd1-461a-9857-8e9cbf14bf49.mproxy.io
Address: 10.0.0.6

Here I am testing against Google’s public DNS server (8.8.8.8) :

pi@rpi3bp:~ $ nslookup 6d8e9885-afd1-461a-9857-8e9cbf14bf49.mproxy.io 8.8.8.8
Server:		8.8.8.8
Address:	8.8.8.8#53

Non-authoritative answer:
Name:	6d8e9885-afd1-461a-9857-8e9cbf14bf49.mproxy.io
Address: 10.0.0.6

So the record is there and valid for the IP address 10.0.0.6

Monocle

@spicymikey said in Monocle Gateway throws Proxy error at startup on Win2012R2 Server:

Regarding the local IP address. Yes I was throwing crap against the wall in the end and trying everything. Thanks for noticing that. I put it back to the public dns name registered on a GoDaddy DNS server.

No problem and once we have your local gateway working, you can certainly use private IP’s for the cameras.

Bad news is my local camera still doesn’t display the video stream from that camera. Not sure if you need to turn your gateway off first, but I turned mine back on, renamed the rtsp url back to using the public DNS name, and said “Show frontyard camera”. No joy.

I look in the log of the cloud hosted gateway and found your camera requests for “Front Yard”. Basically it was unable to connect to the RTSP port (554) at address frontyard-cam.kressahq.kressa.com. Is port 554 open on your router for public access and routed to the camera on your local network?

Jul 26 15:57:27 ip-172-30-0-231 monocle-gateway[13677]: -------------------------------------------------
Jul 26 15:57:27 ip-172-30-0-231 monocle-gateway[13677]: INITIALIZE RTSP STREAM:  Front Yard
Jul 26 15:57:27 ip-172-30-0-231 monocle-gateway[13677]: -------------------------------------------------
Jul 26 15:57:27 ip-172-30-0-231 monocle-gateway[13677]:  - NAME  : Front Yard
Jul 26 15:57:27 ip-172-30-0-231 monocle-gateway[13677]:  - LABEL : PRIMARY
Jul 26 15:57:27 ip-172-30-0-231 monocle-gateway[13677]:  - URL   : rtsp://frontyard-cam.kressahq.kressa.com:554/h264Preview_01_sub
Jul 26 15:57:27 ip-172-30-0-231 monocle-gateway[13677]:  - UUID  : STREAM:64214234-714b-4f8d-8c7d-b381716d4957
Jul 26 15:57:27 ip-172-30-0-231 monocle-gateway[13677]:  - SESS  : c3cf62fa-b8b8-4465-97f1-7872cac523c4
Jul 26 15:57:27 ip-172-30-0-231 monocle-gateway[13677]:  - MODIF : Fri Jul 26 2019 15:54:40 GMT+0000 (Coordinated Universal Time)
Jul 26 15:57:27 ip-172-30-0-231 monocle-gateway[13677]:  - TAGS  : @tunnel
Jul 26 15:57:27 ip-172-30-0-231 monocle-gateway[13677]: -------------------------------------------------

Jul 26 15:57:28 ip-172-30-0-231 monocle-gateway[13677]: 2019-07-26T15:57:28.385Z [INFO]  [107.130.182.161:48074 <BJglXiuzH>] RTSP CLIENT SOCKET CONNECTED
Jul 26 15:57:28 ip-172-30-0-231 monocle-gateway[13677]: 2019-07-26T15:57:28.513Z [INFO]  [107.130.182.161:48074 <BJglXiuzH>] RTSP CLIENT ATTACHED TO STREAM: Front Yard (STREAM:64214234-714b-4f8d-8c7d-b381716d4957)

Jul 26 15:57:58 ip-172-30-0-231 monocle-gateway[13677]: 2019-07-26T15:57:58.514Z [WARN]  [107.130.182.161:48074 <BJglXiuzH>] RTSP CLIENT SOCKET TIMEOUT
Jul 26 15:57:58 ip-172-30-0-231 monocle-gateway[13677]: 2019-07-26T15:57:58.519Z [WARN]  [107.130.182.161:48074 <BJglXiuzH>] RTSP ENDPOINT SOCKET TIMEOUT [107.130.182.161:48074 <BJglXiuzH>]
Jul 26 15:57:58 ip-172-30-0-231 monocle-gateway[13677]: 2019-07-26T15:57:58.519Z [INFO]  [107.130.182.161:48074 <BJglXiuzH>] RTSP ENDPOINT SOCKET CLOSED [107.130.182.161:48074 <BJglXiuzH>]
Jul 26 15:57:58 ip-172-30-0-231 monocle-gateway[13677]: 2019-07-26T15:57:58.575Z [INFO]  [107.130.182.161:48074 <BJglXiuzH>] RTSP CLIENT SOCKET CLOSED
Jul 26 15:57:58 ip-172-30-0-231 monocle-gateway[13677]: 2019-07-26T15:57:58.575Z [INFO]  [107.130.182.161:48074 <BJglXiuzH>] RTSP CLIENT DETACHED FROM STREAM: Front Yard (STREAM:64214234-714b-4f8d-8c7d-b381716d4957)

You mentioned about this DNS entry you are creating, and i read that white paper you suggested. Your hunch is correct. I tried to ping that FQDN and it failed. Where are you trying to create this DNS entry? If you are putting it on your own public DNS server then I’ll need to add that to my DNS search list. if you are trying to inject it into my local DNS server, well, thats not going to work. You don’t have authority. I guess we have narrowed it down and this DNS entry issue must be the problem! Give me more direction on this

Its registered to public upstream DNS servers, so the DNS record should propagate to all public and ISP DNS servers. No, not trying to register anything into your private DNS – in fact, per Amazon’s requirements it must be a public DNS record.

Thanks. Robert

spicymikey

I do run part of my business from home and I have some test servers with company IP on them, so I do indeed have things locked down with a commercial grade Cisco ASA firewall. But i confirmed the rtsp port is opened for all devices behind the firewall.

I think it is the DNS issue you suggested ealier. An NSLookup to that domain name fails from inside my network. But I believe you when you say it is there. In fact, I checked it from my phone on an LTE connection and it did indeed respond with 10.0.0.6.

I have to dig some more, unless you have some hints. It indeed could still be the ASA blocking this local IP address. This PC has a publicly registered Static IP. Is there a way to use the public IP on your DNS entry and let my firewall do the NAT for us?

Monocle

@spicymikey

I do run part of my business from home and I have some test servers with company IP on them, so I do indeed have things locked down with a commercial grade Cisco ASA firewall. But i confirmed the rtsp port is opened for all devices behind the firewall.

Both RTSP and HTTPS (443) need to be able to route on your local network. The Alexa devices will connect to the gateway instance on port 443 and the gateway will connect to your IP cameras on their RTSP ports (554 by default).

I think it is the DNS issue you suggested ealier. An NSLookup to that domain name fails from inside my network. But I believe you when you say it is there. In fact, I checked it from my phone on an LTE connection and it did indeed respond with 10.0.0.6.

I agree. At least this is the first roadblock we have to overcome before trying to diagnose/test anything else.

I have to dig some more, unless you have some hints. It indeed could still be the ASA blocking this local IP address.

I suspect its your DNS server for your network. It probably secures against DNS REBINDING (DNS hosts resolving to local IPs), especially if its a commercial system.

This PC has a publicly registered Static IP. Is there a way to use the public IP on your DNS entry and let my firewall do the NAT for us?

Absolutely. As long as port 443 is exposed to the Internet, you can override the default detected IP address using a custom configuration properties file.
See: https://monoclecam.com/monocle-gateway/custom-configuration

Basically you will change the rtsp.register.host line to rtsp.register.host=X.X.X.X where X.X.X.X is your public IP address. When you restart the service, it will register your public IP address for that DNS host. It may take a few minutes for DNS propagation changes to occur. By default there is a 5 minute cache expiry on these DNS records. Then after a few minutes, do a nslookup on the host and see if its working for your public ip.

Thanks, Robert

Monocle

@spicymikey

I went ahead and disabled the cloud gateway server to avoid any conflicts or confusion for further testing.
We know the Alexa devices on your network will work with a publicly resolvable gateway server and we did validate the demo stream is working using this hosted gateway. So I think we are to the point of just getting your local gateway working now.

Thanks, Robert

spicymikey

The eagle has landed. :)

You had posted a few messages that I had to review. One of them suggested adding a local DNS entry to resolve this name. I did that and the stream is now displaying.

Thanks so much for the help. I would not have been able to navigate through this without your help. I’m still thinking the problem is the ASA rather than the local DNS server. The DNS server is a non-authoritative server for our company domain name. I use it to resolve devices that must be accessible from inside and outside the firewall. It is programmed to just pass requests out to 8.8.8.8 for any zones it does not manage. Clearly myproxy.io was not on my DNS Server so I’m not sure how it could have interferred. BUT, I will pursue further to find out the difinitively what blocked us.

Unfortunately, I doubt the average person has a local DNS server running behind their firewall. So creating a Primary Zone on my local DNS server for your MyProxy.io, and adding an A record for that FQDN, will not be an option for most.

Monocle

@spicymikey

Great News! Glad to hear its up and running!

These more complex networking issues mostly come into play with more sophisticated and commercial (secure) network environments. The typical home user does not often have this issue unless they are running a custom/advanced router like PFSense or Fritzbox. And those types of routers often have some provisions for overriding DNS hostnames.

It’s perfectly fine to leave it as-is with the custom DNS entry – the only thing to remember is to update it if you change the token file or if the local IP changes. Of course figuring out the root cause sounds like a fun little exercise just to fully understand it :-)

You should also now be able to go back to using the local IP for the camera streams and not have to expose it over your Internet connection.

Thanks, Robert

spicymikey

If I figure out the root cause I will post it here for you and others.

I actually don’t normally use the local IP. I just did that for testing. The goal is to be able to monitor the property from anywhere, not just inside the house.

I just made a $50 donation on your home page as a thanks for staying with me on this one. Thanks again!