Strange Google Pixel DNS things

I’ve had this off and on issue for a while in which images and videos don’t load on Twitter with my Google Pixel 3 on both Android 9 and Android 10. If I wait long enough—maybe 60 seconds–the current timeline populates and things generally work.

This makes Twitter even more annoying than its baseline. I often use this as an excuse just to stop using Twitter for the hour or night.

Recently, I noticed that images in the Guardian app weren’t loading either. The full text of the articles would come in, but there were blank boxes everywhere. These aren’t always necessary for the story, but that they’re missing is a little distracting and makes me wonder what else isn’t working on the phone.

While I was out of the house and on LTE today, I noticed that images and videos in Twitter and the Guardian were loading fine. Something finally clicked in my head and I decided there must be some kind of funky problem with my network setup at home.

I’ve had the Cloudflare DNS servers configured on our router for quite a while and haven’t given them too much thought since they seem to work just fine. Today I made a temporary change to Google’s public 8.8.8.8 and 8.8.4.4 DNS servers to see if it made an impact. Sure enough, Twitter images and videos started loading immediately.

I then changed to OpenDNS’s public 208.67.222.222 and 208.67.220.220 DNS servers and the issue came back—no Twitter images or videos. Switched back to Google’s DNS and they started working.

The strange part at this point is that Guardian images still aren’t coming in.

So then I switched back to the DNS my router receives from the ISP and everything immediately started working. Imagine that!

This is great and all, but ISP DNS is generally disgusting. When I try to go to a domain that doesn’t exist, the request is automatically routed to a “non-existing domain landing service” that is branded by Spectrum, Time-Warner, and Charter. That there’s not just one company name associated shows how careful the friendly conglomerate is with branding (and customers’ data).

An opt-out link is at the bottom of the page, but it doesn’t actually work.

So what to do. None of this has any impact on our MacOS laptops. I should do some actual testing, but the DNS lookups still seem faster to 1.1.1.1 than anywhere else. I tried the Cloudflare 1.1.1.1 app on the phone to do DNS over TLS, but it has trouble with Twitter images and videos too. Android doesn’t appear to let you override the DNS, so all requests are made through the router. I’m guessing that ends up being part of the real issue.

I’m going to go with the Google DNS servers for now and see how it feels for the next few days. Then I guess start diving into Netgear Orbi forums to figure out how it could be screwing things up. 🤷🏼‍♂️

Update, 6:55pm November 24, 2019:

A couple issues combined here:

  • There should be a way to manually set DNS in Android, but that doesn’t appear to be available to me. For this reason, the DNS is set to 192.168.1.1 rather than any DNS server I configure on the Netgear Orbi.
  • It should be possible to tell the Netgear Orbi to pass DNS servers via DHCP other than its own IP. The Orbi has dnsmasq running, so it apparently assumes that will be the best option and now disables all others.

So now I need to figure out how to get the DNS actually set to the best provider on my phone without downloading one of those shady DNS apps.

Update, 7:11pm November 24, 2019:

From what I can tell, Android 10 no longer allows you to set custom DNS for a specific network. But! There’s a private DNS option available in the general network settings that allows you to configure DNS over TLS.

I first set this to 1dot1dot1dot1.cloudflare-dns.com, but the issue persisted. I then set it to dns.google and images started loading immediately in both Twitter and the Guardian. Videos are still a little funky, and I’ve found a few images that don’t load, but this may be a good short term fix.

The long term fix needs to be making the router not improperly handle DNS queries for the network.

Update, 7:29pm November 24, 2019:

I’m irritated enough at the router for running its own dnsmasq instance that I’ve now reached the hack or replace stage of router ownership.

Update, 8:09pm November 24, 2019:

I have shell access to the router and have updated the dhcpd configuration to send 8.8.8.8 as the DNS server to DHCP clients rather than the router’s IP.

This is excellent because it solves my irritation that the router was running dnsmasq poorly. This is less excellent because my actual issue with images is still there. 😂

Update, 8:13pm November 24, 2019:

There’s a chance that things are fixed? I’m not really sure, but I should switch to something else. I’ll add an update after a bit once I’ve actually had a chance to use it.

Various Networking Configurations in VVV

I dug in to some different configurations in VVV today and decided to write them up as I went. This will be posted in some form to the VVV wiki as well. There are other networking configurations available in Vagrant, though I’m not sure that any would be useful in development with VVV.

I would recommend using default settings for initial provisioning as things can get quirky inside the VM when trying to access outside sources. Run vagrant reload to process any network configuration changes.

Private Network (default)

config.vm.network :private_network, ip: “192.168.50.4”

This is the default configuration provided in VVV. A private network is created by VirtualBox between your host machine and the guest machine. The guest is assigned an IP address of 192.168.50.4 and your host machine is able to access it on that IP. VVV is configured to provide access to several default domains on this IP address so that browser requests from your host machine just work.

Outside access from other devices to this IP address is not available as the network interface is private to your machine.

Port Forwarding

config.vm.network “forwarded_port”, guest: 80, host: 8080

One option to provide other devices access to your guest machine is port forwarding. Uncommenting or adding this line in VVV’s Vagrantfile and then running vagrant reload will cause any traffic on port 8080 directed at your host machine to instead communicate with port 80 on the guest machine.

This configuration will work with private or public IP configurations as it deals with port forwarding rather than the IP of the virtual machine itself.

An immediate way to test this once configured would be to type your host machine’s IP address into a browser followed by :8080. With port forwarding enabled, something like http://192.168.1.119:8080 would bring up the default VVV dashboard.

Of course, this doesn’t do you much good with the default WordPress sites, as you’ll be stuck adding port 8080 to every request you make.

The easiest hack around this is to setup port forwarding on your router. Point incoming requests for port 80 to port 8080 on the IP address of your host machine. Requests through the router will then traverse ports 80 (public IP) -> 8080 (host) -> 80 (guest) and your development work can be shared with devices inside and outside of your network.

Say my router’s public IP is 14.15.16.17 and my computer’s local IP is 192.168.1.100.

  • Enable port forwarding in Vagrantfile.
  • Configure router to forward incoming port 80 to port 8080 on 192.168.1.100.
  • Visit src.wordpress-develop.14.15.16.17.xip.io on my phone, connected through LTE.

There are other things you can do on your local machine to reroute traffic from 80 to 8080 so that it forwards properly without the use of a router. Sal Ferrallelo has posted steps to take advantage of port forwarding directly in OSX using pfctl.

Public Network

config.vm.network “public_network”

Replacing our default private network configuration with a public network configuration immediately provides access to other devices on your local network. Using this configuration without specifying an IP address causes the guest machine to request an address dynamically from an available DHCP server—likely your router. During vagrant up, an option may be presented to choose which interface should be bridged. I chose my AirPort interface as that is what my local machine is using.

==> default: Available bridged network interfaces:
1) en0: Wi-Fi (AirPort)
2) en1: Thunderbolt 1
3) p2p0
4) bridge0
5) vnic0
6) vnic1
    default: What interface should the network bridge to? 1

Once the guest machine receives an IP address, access is immediately available to other devices on the network.

  • vagrant ssh and type ifconfig to determine the IP address of the guest – mine was 192.168.1.141.
  • Visit src.wordpress-develop.192.168.1.141.xip.io on my phone, connected to the wireless network.

To me this is most desirable as it provides access to devices on the local network, not to the outside. If you are using public wifi or another insecure network, be aware–this does open your machine up to other devices on that network.

config.vm.network “public_network”, ip: “192.168.1.141”

The same configuration would be available without DHCP by specifying the IP address to use. If you know what subnet your network is on, this may be a shortcut for providing access without having to use ifconfig inside the guest machine.