The BIG problem

In April 2023 the BIG problem that I feared so long finally arrived.

GTT (gtt.net my ISP) sent me a >letter<. The letter does not mention it, but GTT has filed for bankruptcy protection. The letter is full of things that I do not fully understand, but the bottom line is: "We no longer want to do business with you. You have a few months to get a new ISP, then we will no longer provide service at your location."

This page documents what I did. In short, I asked advice, got a coax internet connection from Spectrum and a VPS from Afterburst, and went on with that until March 2026. At that time Afterburst in turn sent me notice that they would discontinue service.

Table of Contents

The Lossage List is a log of various problems over the years related to my internet connection. This final problem goes on the log of Bigger Problems.

I have had the same IPv4 address for almost exactly 20 years and a lot of things depend on that. There are three domain names that point to that address. There is a name server, a mail server, and a web server running at that address. This page is served from that address, as well as all my personal email and a mailing list.

Here is a block diagram of my home network.

I don't know how many of those things can be kept working with a new ISP.

I don't have much hope of keeping the IP address with a new ISP, but I would like to keep my domain names and continue to send and receive mail with addresses in those domains, and to be able to put up web pages in those domains with a minimum of censorship and advertising. I don't need the name server if there is another way of doing this.

Actually, changing the IP address would be a small change. The big change is that the IP address I have sends packets to a machine in my basement. It seems that renting a Virtual Private Server "in the cloud" is the way the internet works now. It would be nice if the programs and configuration that I have now could be just run there, but I fear I will need to learn a whole new system.

I live in Worcester, about a mile from WPI.

I will spend a few days asking for advice, and post updates here if I can.

The plan forms

Since I knew that GTT had filed for bankruptcy protection, I assumed that was the root cause of this. I asked some questions on the Worcester Linux User Group mailing list which led me to doubt that. I then asked two more friends.

I am beginning to think that basement hosting is no longer possible. The current tentative plan is to move one of my domains to a ``cloud server'' then use the DSL connection as long as possible to test it by looking at web pages and sending email between domains.

WLUG advice

This is an edited conversation that happened on the Worcester Linux Users Group mailing list.

WLUG> The issue is Verizon.

As soon as I saw that, I started to write a reply that said:

That's provably false. The letter came from GTT. Verizon is not under discussion.

While I was editing that to make it more polite, I read on:

WLUG>  We were officially not getting our DSL "service" from Verizon but it was their copper. Verizon did not even notify our ISP and just disconnected the line (in Marlboro). We of course found out when our equipment went offline.

Now the question becomes: "How much do we trust Verizon?"

That got me thinking. The letter from GTT does not mention Verizon, but why would they? Blaming Verizon might look (to the customer) like a lame excuse; it might look (to Verizon) like a legal cause of action.

It doesn't need to be a conspiracy, sometimes quarrelling parties, each for their own reasons, don't feel motivated to explain things to us.

Now the skills I learned in grad school, reading Kant's "Critique of Pure Reason", begin to suggest more questions. What did they actually tell me, and what have I added myself?

The GTT letter is clearly a form letter. It might have been written years ago, to be automatically filled in and sent to anyone they plan to disconnect for any reason anywhere in the world. It says nothing at all about why. I just assumed they decided there was not enough money to be made by charging a monthly fee to leave everything the same.

WLUG>  So I suspect that it's not GTT that is really providing DSL but more likely Verizon.

This is true. GTT pays Verizon for local DSL delivery over copper wire. Every time I report a problem to GTT, they send Verizon to my house.

WLUG>  And since VZ is trying to get out of the copper business that is why your dilemma. However Verizon is working to roll out Fios in the city so maybe they will get your neighborhood before you're disconnected.

If this were true, my plan would be to order FIOS (Fiber Optic) internet connection from Verizon and run it in parallel with DSL for a few months. Then I could get set up with the Fiber while continuing to use DSL for email, downloading software, seaching the web for answers, etc.

I am afraid of being totally without internet while trying to get the new connection working.

Unfortunately, the Verizon web site says FIOS is not available in my area. But when I tell their chatbox on their web site of my fears of a coverage gap, it/she/he seems suspiciously knowledgeable about my problem and tells me not to worry.

There won't be any gap. You will get an email prior to 30 days earlier to the shut down to port your services to Verizon. — Verizon

WLUG>  I'm not sure where your web servers are hosted

They are hosted on a Redhat Fedora Linux Intel Pentium in my basement. Linux is about doing it yourself. I don't have a web server host, just Apache running on Linux, not up in the cloud, down in the basement. See the block diagram linked near the top of this page.

More WLUG Hosting Advice

This a list of some out-of-context comments from WLUG.

Hosting Advice 1

Hosting Advice 2

I used to have a direct line into my house and all my servers in one of the rooms of my house. The ISP changed hands *many*, *many* times. Eventually, they didn't like me very much because the actual hardware into the house was quite old and slow and being in a residential area, if things broke elsewhere, they'd steal parts from the substation nearby, as I was the only place being serviced. Then it would take them awhile to replace them. (I found this out from one of their more honest repair people).

Anyway, at some point it became both cheaper and more reliable to move everything to 'the cloud' and just have a comcast line coming in. The household gets cable, the business gets internet and my business phone.

I currently rent two Linux servers 'in the cloud', from a company called Afterburst.
ME>  https://afterburst.com is a link to Afterburst.
I do know that a company called RackSpace is quite popular, but I have not used it.

I pay $15/month for one of my servers, and $120/year for the other. One has more CPU and more disk space than the other.

They give you a console login, where you can reboot the machine from. I generally try to not login via the console. Instead I usually ssh in, then sudo when I want to mess with the system. I run DNS, mailman, websites, and a mail server. I lost patience with sendmail long ago, and found out about SurgeMail, which I do like quite a bit, which lets me set up my customer domains with email accounts. You license SurgeMail by number of domains. I either have 50 or 100. But it also needs renewing once a year to have the latest updates.

SurgeMail/SurgeWeb also has an ability to run mailing lists, but as of yet I have not explored this capability. Of course, you can just continue to use what you've been using ... the cloud Linux servers are full Linux.

I do get a static IP via my cable, but it didn't seem viable for supporting my customers. I do use it sometimes for other things.

In addition to saving money from my old setup, I have better reliability. My road tends to lose power several times a winter. There were a few times that were for 10 days. The big server farms have backup power and what have you. My customers are happier. And they have more powerful connectivity.

Well, I hope some of this helps. Feel free to ask questions. Maybe I'll be able to answer them. who knows? good luck!

ME>  There must be some kind of connection to the house. What is it? You can't just go to the cloud all by yourself.

I get my connection via my comcast cable connection now.

ME>  Is the console on a web page? What software runs on the home machine?

Yes, the console is via a webpage, at least for Afterburst. Don't know how Rackspace works it. At home I run my various laptops -- Mac and Windows personal laptops and ones owned by companies I do contracting with. And sometimes I run customer Linux boxes. The laptops get out to the internet via the comcast router. The comcast router deals out DHCP internal addresses, and I give the Linux boxes static internal addresses. Anyone who visits can use the comcast WiFi to get out to the internet with their phone or laptop if they know the password.

Internet Hosting

Here is a summary list of the Hosting suggestions given above.

Hardware and Wires

GTT

I tried to get some more information by logging on to the GTT customer web page (called Ethervision). It seems that according to the GTT customer database, I have been disconnected for five years, and three months before the request. As far as I can tell, nothing relevant happened between September and December 2018.

2023-04-24(Mon) 23:16:47EDT:

Log into ethervision.
choose Services => Disconnects
it says:
   1143703 | 10224655-5 | Colocation-Worcester,US |(My Address)| Disconnected
=> Click that line
it says:
   GTT Service ID - 10224655-5
   Service Status - Disconnected
   Signed Disconnect Request Received: 2018-12-05
   Requested Disconnect: 2018-09-15

Verizon

The Worcester Telegram & Gazette of Wednesday, April 26, 2032, has an article headlined ``Verizon preparing to make connections''. It seems to be a jumbled compendium of quotes from various people disagreeing about various things that the reporter does not understand. I gather (I think) that Verizon and the City are quarrelling about who will install ``broadband'' in Worcester. Various people say it is happening Real Soon Now, or that it is impossible. A partcularly inscrutable quote from a City Manager:
… the city not owning its electrical grid puts it at a disadvantage … Buying the grid from National Grid would be a tough negotiation.
I have no idea why owning the electical power grid would be an advantage to one who installs fiber optic cables. Are they fighting about who gets to hang cables on the poles? — More likely they are fighting about who gets to pull cables through the underground conduit.

Spectrum

I have been annoyed for years by Spectrum ads that come in the mail with "Do Not Bend" written on the front. Inside is a totally worthless card. Nevertheless:

2023-09-08: I went to the "Spectrum Store" in Worcester and got a self-install kit. It contains

I brought it home and plugged it in. I thought I would have to contact Spectrum to turn it on, but after a minute or two the Online status light came on.

The final step on the printed instructions is

D Activate your modem by visiting spectrum.net/selfinstall on your computer or smartphone and following the onscreen instructions.
That didn't work. I could see the web page; it's slow. I couldn't enter any data. I saw errors messages flash by that said "something is wrong" before disappearing. It might be because the DSL connection died around that time.

The question arose whether Spectrum wants me to use the new cable modem to activate? The answer is "No". The printed instructions say to do it on a smartphone. If I try to connect with a wireless computer, it can see the WiFi SSID, but using it results in "No Internet", and "Can't Find IP address of spectrum.net". So it seems that the new cable modem can not be used until after it has been activated.

Software and Configuration

Before moving all this to "the cloud", I had better upgrade to a newer Operating System. It might be difficult to get a virtual machine running Fedora Core release 3 (release 2004-11-08; end of life: 2006-01-16), which is what I was running.

This page is now served by a newer machine running Debian Linux version 11.7, bullseye (release 2023-04-29).

2023-09-09: Cancel that. I thought I would move to the new machine; I got it working, but moved back. The new plan is to use the old machine for DSL and new one for coax. Until coax works this page is served by a machine with Tandy monitor and Fedora Core 3. I'll miss it, but it's not gone yet.

ssh

Years ago I used telnet, now I use ssh to connect between computers on my home network. I use it because it was installed and just worked with the Linux distribution, Fedora or Debian. The Secure Shell Daemon (sshd) on the old (Fedora) system only accepts connections from the internal network, so I was not too careful with authentication.

It looks like I can no longer be so sloppy if it is to be hosted in the cloud. Whatever hosting I use, I will need ssh to connect to it. I should know something about how that works.

I recently installed a new Debian system on a computer I call fcs21, and tried to connect to a very old Fedora system on fcs19 and got the following complaint when I tried to connect from 21 to 19: 

  kwright@fcs21:~$ ssh fcs19
  Unable to negotiate with 192.168.1.118 port 22: no matching key exchange method found.
    Their offer: diffie-hellman-group-exchange-sha1,
		 diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
I made this file, which fixed everything 
  root@fcs21:~# cat .ssh/config 
  Host fcs19
      KexAlgorithms +diffie-hellman-group1-sha1

Afterburst

I sent an email to Afterburst. Notice that this entire exchange takes place in one day. 
  To: Afterburst
  Subject: Moving a server from my basement to Afterburst
  Date: Mon, 04 Sep 2023 13:16:42 -0400
For 20 years I have had the same IP address, with IP packets transferred through a DSL line to a modem in my basement which connects to Ethernet inside. I then have an Intel Pentium running Debian GNU/Linux 11, which runs Postfix smtpd, and Apache2 httpd.

Unfortunately, my ISP intends to cut the wire at the end of September 2023. My current plan is to get "home internet" (ie a web browser with no IP address) somewhere else.

How much of my current configuration can I just upload to Afterburst and manage through a web browser? 

    From: Afterburst
    Date: Mon, 4 Sep 2023 18:53:28 +0100
    Subject: Re: Moving a server from my basement to Afterburst
We don't provide a web interface e.g. for email/web hosting -- just control over a virtual machine. Essentially boot/reboot/emergency console, reset root password, mount ISO's, reinstall etc.

You get full root access over ssh, so you can manage the system that way - you can install web control panels for websites, email from there if you wish.

Our services come with dedicated public ipv4 and ipv6 addresses (1x ipv4 + /64 ipv6).

We'd recommend installing the system and configuring from scratch -- but a lift/shift of a physical systems disk onto a KVM VPS is possible. We advise against doing this unless there's no other choice - but it can be done
… there follows a short discussion of using dd to copy the whole disk. 

    To: Afterburst
    Date: Mon, 04 Sep 2023 16:54:42 -0400
I don't want to use dd
Afterburst writes:
We don't provide a web interface e.g. for email/web hosting -- just control over a virtual machine.
That's good, I am not looking for email/web hosting, just moving the servers, that are now running on hardware at the end of DSL, to a virtual hardware.
Our services come with dedicated public ipv4 and ipv6 addresses (1x ipv4 + /64 ipv6).
That sounds very good. I would just install Debian on virtual server and copy some of the configuration files (with needed changes). For example: 
       root@fcs21:~# tail -n 4 /etc/network/interfaces
       iface  enp5s9 inet static
	      address 66.92.74.188/24
	      gateway 66.92.74.1
Can I just replace my old beloved address (66.92.74.188) with one chosen by Afterburst and expect Postfix to process SMTP commands at the new address (assuming DNS MX records point to new address)? 
     From: Afterburst
     Date: Mon, 4 Sep 2023 22:53:03 +0100
Yes, that should largely be it. For outbound email we would also recommend at minimum setting up FCrDNS (matching forward and reverse DNS - contact us via ticket after configuring the forward record) and SPF (DNS record) with your assigned IP address.
… FCrDNS is described at
https://en.wikipedia.org/wiki/Forward-confirmed_reverse_DNS
and SPF at
https://en.wikipedia.org/wiki/Sender_Policy_Framework

Spectrum Configuration

2023-09-11: I plugged ethernet cable between WiFi router and enp5s9 on fcs21, and changed interfaces so that it uses DHCP. Then 

     
	root@fcs21:/etc/network# ifup enp5s9
        ... ... ...
	DHCPOFFER of 192.168.1.30 from 192.168.1.1
	DHCPREQUEST for 192.168.1.30 on enp5s9 to 255.255.255.255 port 67
	DHCPACK of 192.168.1.30 from 192.168.1.1
	bound to 192.168.1.30 -- renewal in 19618 seconds.
That looks good, how do I set the routing table?

I think it means that the WiFi router has address 192.168.1.1. That could be a problem, because in the old setup [1], the default route to the internet and all the servers, including the name server, were on one machine, and I gave it one address (192.168.1.1).

That is a problem for the machine in the middle (fcs21) because it must accept connections to servers on one address, but forward packets to the other. Any other machine can forward to it, but it shouldn't forward to itself.

On the other hand, to every other machine on the LAN, fcs21 is the default route to the internet. Is there a way for it to be 192.168.1.1 on the internal net and still forward packets to the WiFi router? Or should I change all the other machines to default route to 192.168.1.10 or something? How do I choose an address that will never conflict with one chosen by the DHCP server in the router?

A New Plan

The old plan was to mimic the old setup, but with GFRS (Gateway, Firewall, Router, Server) using DHCP instead of a static IP address to connect to the internet.

The above questions, plus some advice from networking experts, convinced me that it won't work that way. Instead, the address range 192.168.1.0/24 will be under control of the WiFI router. A machine that connects to the internet must use DHCP to get an address from the router. Static addresses will be changed to 192.168.2.0/24.

I am working on plan B.

Good HTML 4.0
This page is standard validated HTML 4.0.
( Checked 2023-09-12 )
Creative Commons License
This work is licensed under a
Creative Commons Attribution-Share Alike 3.0