Dear imonitor users, potential users, former users, and interested parties:

Summary for those of you new to this newsletter: 

I am working on a project on Internet [and local network] monitoring, and have a small device which I can deploy to help measure/monitor your Internet service.  I have about 20 "guinea pigs" deployed and am always interested in expanding my "customer" base.  I initially targeted the service to users on the mountain in Jasper GA (windstream ISP), but it is applicable universally, and I have "customers" across the country on Windstream, ATT, Spectrum, Hughes, Comcast, Qwest, CenturyLink, and several others.  I am always interested in getting "customers - guinea pigs" on other ISPs.  Please read on if you are interested.  I am particularly interested in comments and suggestions from my more "nerdy" customers!! 

I have a spare gadget for rent [free]!   Let me know if you interested.  OR..... you can purchase an raspberry pi 3B or 3B+ and I can send you a microSD image!  Nerds might be interested in this alternative.  Currently, I can only offer an image [3B or 3B+, not 4B at this time], and do not have an rpm or a deb to add the service.  But you can receive the image and then use the rpi for your own purposes - just don't mess with the scripts [guinea pigs are certainly encouraged to use existing rpis for their own -additional- purposes].  You can always opt out of the active management [stats are always available at the web page].  But since the imonitor is still under development, it is best to remain "in the program" for the upgrades, or enable the mgmt when asked.  Here is the raspberry pi 3B+ I usually order:

https://www.amazon.com/gp/product/B07BLRSKBV/ref=ppx_yo_dt_b_asin_title_o00_s00?ie=UTF8&psc=1

As usual, I am extremely grateful for the use of your ISP connection to develop this service across many ISPs.   It has been invaluable, and much fun. 

As always, you can refer to the main information page at https://johnloop.com/imonitor.html  There are images of the web site, email, plots, etc. 

There is a new "quick intro" doc at https://johnloop.com/imonitor/QuickManual.pdf  [also attached to this email]

Newsletter 12-1-2019  previous newsletter https://johnloop.com/imonitor/newsletter9-1-2019.html

There is news in these areas

1. The Rpi does a scan of your local network at 7PM and reports "new" and "gone" hosts compared to 7PM day before.  This is done using the host IP address.  There is no universal way to "ID" IP hosts -getting their name e.g.- so this is useful only if you understand what is on your network.  I will be adding a scan of these active hosts to keep you alerted as to the services open on your network.  If you are running these on wifi [especially], these are potential intrustion avenues for lurking hackers.  Wifi traffic activity [unlike ethernet] is completely visible up to 200 feet from your router, and can be compromised by determined hackers.  [this scans ONLY the network that the rpi is on.  Some customers have added networks behind their routers]

2. The historical offline plot now works correctly, and will be of more value going forward.  The historical speedtest plot is working, so you can see an archive of your speedtest results.   -available via the email links or on the rpi web page.

3. The archive plots of your ICMP and TCP ping responses is a wonderful way to gauge longterm performance.  -available on the rpi web page.

4. The alerts now seem to work :)  you will get an email alert for the following:

- a reboot [which may represent power cycle in your house].  Ideally connect the rpi and your router to a UPS! 

- an Internet IP address change for your modem [which can be expected occasionally].  Useful if you have services running and need to get to them remotely.

- for "offline" events, where Internet access is interrupted. 

- for DNS interruptions, which may or may not be offline events.   DNS changes are also reported.

You can turn off all email of on the mgmt config on the web page, or just enable the Saturday status email.  The alerts are quantized in that they "may" not detect alert events lasting less than 5 minutes - such as DNS server or Internet timeouts.  This is probably best to "smooth" the alerts.  You can always see the offline events by looking at the ping plots or the actual alerts on the rpi web page.

5. The Rpis have been updated and upgraded to the latest code.  3B is "jessie."  3B+ is "stretch."

6. Temperature info has added a degF current reading on the rpi web page - as well as yesterday plot.  I can add a temp probe if desired!  This is used by several people to monitor rental properties. -the temp plots are archived on the rpi web page.

7. The emails are archived on the rpi as well.

I continue to fine tune the alert scripts.  I may decrease the quantization to 1 min [from 5 min] since the script is more optimized.  I will be adding a scan of the active hosts on the network [1 above] to try to ID services and summarize them.  This would list both ethernet and wifi services available [if the rpi is logged into your wifi].  Every day you would get a list of the "services you are offering!"

You receive a list of the wifi networks visible on your network [listing your neighbor's as well] - reported in the daily email.  These are the networks visible, and should not be accessible unless you have the wifi key.  The rpi can login to your wifi network as well as the ethernet by providing the SSID/key.  You can do this, or I can do it remotely.  

Latest guinea pig list.  Some of you have asked about my "guinea pigs," where they are, and how their Internet access compares to theirs.  Here is a table.  I have plots for each site at   https://johnloop.com/imonitor/customerplots/customerplots.html -> take a look and compare yours! I am using a representative plot from Aug 30.  I will have examples and pointers about interpreting your plots and data going forward on that page. 

As usual comments and suggestions are greatly appreciated!  I could not do this without the "loan" of your ISP.  For that I am eternally grateful.

John Loop

You are included in this newsletter for various reasons.  If you do not wish to receive it, then just simply respond to me saying such.  No problem. 

Newsletter -9-1 Points repeated here:

1.  I have combined the ICMP ping and TCP ping plots into one plot, called "BOTHYpingresult.png" and have inserted the time on the horizontal axis, making it much easier to gauge daily performance.  This is the plot that is forwarded each day via email.  The previous plots of individual ICMP and TCP pings are still available @ the rpi web page.  This plot is linked in the daily email.  I have attached a sample.  [This shows a particularly crummy connection!]

You can easily compare the plot to previous days' plots to give a feel for day-to-day performance.  Great visual way to do it.  Remember the DATE on the plot actually will actually contain the data for the day before (1/2AM yesterday -> 1/2AM today).

[the one wrinkle about the plot is that it starts at 1AM/2AM yesterday, goes to 12AM today and wraps to the left and goes to 1AM/2AM.  So start at 1AM for TCP pings, 2AM for ICMP pings :-) ].  I may try to clean this up. 

2.  I have created an [Internet] offline record -viewable on the Rpi web page- and a plot.  The record shows the date/times and offline minutes encountered during the year.  [I am still working on the nuances of this plot, so it may be iffy still].  This plot [sample attached - BOTHYping---.png] is linked in the daily email.  I have modified scripts to give a better/more accurate alert when power cycles/offline events/DNS or IP address changes are detected.

Beware - the offline plot does not work well yet.  I have the data, just not the correct plot parameters.  Fixed asap!!

3.  The TCP ping now pings the top 100 US web sites [as default], instead of a fixed site.  The fixed site [selectable] is still used for the "connect" test, but the "tcp ping" test will use either the fixed site [as before], or the top 100 web sites.   "AUTO" enables the top 100 web sites, "LOCK" will revert to use the previous fixed site.  It should give a much better feel for the performance of the "entire" Internet. I consider this a great feature!    

I have attached a sample plot which shows the switch between the fixed target and the top 100 targets [pi25swBOTH--.png] scan.  It results in a "scatter-plot" as the pings work their way around the Internet, much more representative of actual performance.  The first part of the red plot is the fixed tcp ping target, and the second part is when it is pinging the top 100 web sites.  It simply round robins the 100 sites every 100 minutes.

The amount of Internet traffic generated by the Rpi is still minimal - so it will not interfere with your connection.   I will characterize next newsletter.

4.  I have been working to get around the fact that the Rpi has no realtime clock.  This can be difficult.  If possible, to maintain the most valid stats, you should have the Rpi on a UPS with your modem/router. The boot alerts can get confused as try to obtain time. 

5.  Some of you have asked about my "guinea pigs," where they are, and how their Internet access compares to theirs.  Here is a table.  I have plots for each site at   https://johnloop.com/imonitor/customerplots/customerplots.html -> take a look and compare yours! I am using a representative plot from Aug 30.  I will have examples and pointers about interpreting your plots and data going forward on that page. 

I am eternally grateful for allowing me to use your Internet connection to develop this service.  I am hoping it is a more valuable service as time goes on. 

General description of rpi monitor tests:

Bash scripts running on a small computer on your network attempt a connect to a target every minute of every day.  Statistics are reset at beginning of month.  The target is configurable, default is google.com.

The scripts perform a speedtest at 3AM and 3PM.  This is done from the Rpi of course!    

The scripts perform a single ping every minute of every day to a stable Internet IP address.  These are counted and reset at beginning of every day.  There will be an entry for each ping failure.  The ping target is configurable.

The wait time for both the connect and the ping is 15 seconds; no response and a failure is declared.  There are 44640 minutes in a 31 day month.  I don't do quite that many.... housekeeping and such. 

Last minute ping delays

Yesterday ping delay average and plot

Today ping delay plot -from 2AM to when you click the link.

Last hour ping delay plot

Route to IXC

Yesterday temp average [if enabled].  Up/down limits can be set and you can be alerted.

Scan of wifi networks from your location listing all SSID and sig levels, other info

Scan of your Internet connection to check for vulnerability

Scan of your internal network -if enabled

Week's worth of stats available in piweeklytars directory

Internet IP address changes are recorded, and you can be alerted for a change

Boot times of the Rpi, which may represent power bounces in your house.  You are alerted.

A browsable web page [safely on the raspberry pi on your local network] with all these stats available at any time.  Link provided in the email.

Access to your router [gateway] via the web page.

Raspberry Pi can be behind a secondary router.   

There is absolutely no monitoring, capturing of your Internet traffic.  The raspberry pi generates its own to collect stats.  The bandwidth demands on your connection average less than 1Kb/s. 

I can give you access/login to the raspberry pi if you wish. 

I can give you an SD micro with the software if you have your own raspberry pi 3B or 3B+. 

John Loop pccitizen@gmail.com  txt to 706 669 7164  or jdloop@johnloop.com 

https://johnloop.com