Search This Blog

Match a string ending with a dollar sign ($) in PowerShell

I want to match a string ending with a dollar sign ($) (e.g. ‘contoso\john.smith$’) in PowerShell. Using the regular expression (regex) should be simple, like \$$.

However, I run into some problem in PowerShell. I posted my question in Reddit for help. With the comments from the community, I think I finally understand how to handle this issue. This blog post is to summarize my understanding.
  • -match operator uses the regular expression syntax.
  • The scape character in regex is the backslash(\). Normally the regular PowerShell escape character, the backtick(`), should not use in the regex expression. See my next post on using both scape characters (\ and `) in one expression.
  • To match a string ending with a dollar sign ($), the regex should be \$$. The first $ is for the literal $, so it is escaped by \. The second $ is an anchor which matches the end of a string, so it is not escaped by \.
  • However $$ is an automatic variable in PowerShell.
  • When the expression(\$$) is doubled-quoted, PowerShell evaluates the variable $$ first before sending to the regex engine. “\$$” becomes “\{value of $$}” when being parsed by the regex engine. So it returns False.
PS C:\Temp> 'contoso\john.smith$' -match "\$$"
False
  • When the expression(\$$) is single-quoted, PowerShell does not evaluate any variable. The expression \$$ is parsed by the regex engine. So it returns True.
PS C:\Temp> 'contoso\john.smith$' -match '\$$'
True

In the case, I can match a string ending with $ by using the single quote with the expression '\$$'. However, this will not work if the expression includes other variable that should be evaluated. I will post my solution in the next post.

My Pi-hole blocklists

After setting up the Pi-hole on the Raspberry Pi, I start looking for additional blocklists to block as much as unnecessary traffic on my home network. 

Searching on the Pi-hole Userspace, I found many discussions about the blocklists. I want to share my blocklists on this post, so you can save time.
My goals are to use the blocklists that
  • Are from the reliable source
  • Are updated regularly
  • Provide additional filter categories that are safe for family
  • Are free of charge
As mentioned in my “Select Upstream DNS server for Pi-hole” post, I may be able to use OpenDNS or CleanBrowsing DNS as my upstream DNS server to achieve the same goals without using the blocklists. But these blocklists become necessary after I set up DoH with Cloudflare DNS because Cloudflare DNS doesn’t provide content filtering.

The following are the blocklists in my Pi-hole. You can copy and paste them under Pi-hole’s admin page, Settings, Blocklists, and click “Save and Update”.

At the time of this writing, these lists block over 3,900,000 unique domains. I have to admit some legit domains also being blocked, e.g. the Target.com's Weekly Ad. But they can be fairly easy to identify and add to the whitelist. I will share in a future post.

Please provide your feedback if you have a better list or issue with my list. I will keep the list up-to-date.

https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
https://s3.amazonaws.com/lists.disconnect.me/simple_tracking.txt
https://s3.amazonaws.com/lists.disconnect.me/simple_ad.txt
https://blocklist.site/app/dl/ads
https://blocklist.site/app/dl/crypto
https://blocklist.site/app/dl/drugs
https://blocklist.site/app/dl/fraud
https://blocklist.site/app/dl/fakenews
https://blocklist.site/app/dl/gambling
https://blocklist.site/app/dl/malware
https://blocklist.site/app/dl/phishing
https://blocklist.site/app/dl/piracy
https://blocklist.site/app/dl/porn
https://blocklist.site/app/dl/proxy
https://blocklist.site/app/dl/ransomware
https://blocklist.site/app/dl/redirect
https://blocklist.site/app/dl/scam
https://blocklist.site/app/dl/spam
https://blocklist.site/app/dl/tracking
https://dbl.oisd.nl/

Set up Cloudfared DoH for Pi-hole

I would prefer using DNS over HTTPS (DoH) to increase privacy and security. Mozilla Firefox is the first web browser implementing DoH, and many other browsers follow. But how can I utilize DoH for the non-browser DNS request or enable DoH on each device that connects to my home network?

Pi-hole has a document to configure DNS-Over-HTTPS. To automate the install and configure Cloudfared on a Raspberry Pi running Raspbian, I create a bash script. I also add the steps to lock down the cloudfared account.

Here are how set up Cloudfared DoH using the script.
  • Make sure the Pi-hole is set up on your Raspberry Pi. See my post “Set up Pi-hole on a Raspberry Pi”.
  • Download the script from my Github on your Raspberry Pi
    • curl -O https://raw.githubusercontent.com/sfitpro/pi-hole/master/setup.cloudflared.doh.for.pi-hole.sh
  • Grant the execution permission to the script
    • chmod +x setup.cloudflared.doh.for.pi-hole.sh
  • Run the script
    • sudo ./setup.cloudflared.doh.for.pi-hole.sh
  • Configure the Pi-hole to use custom Upstream DNS server 127.0.0.1#5053
If all the setup is correct, you will notice the DNS queries on Pi-hole are answered by localhost.


Select Upstream DNS server for Pi-hole

Once the Pi-hole is up and running, the first thing I want to configure is its upstream DNS servers. The upstream DNS servers can provide additional filters (e.g. adult-related sites, social networking sites, etc) that are not included in the Pi-hole default installation.
I have been using OpenDNS as my home router's upstream DNS servers for a long time. I sign up an OpenDNS account so I can customize the kind of content to block or the sites to be whitelisted. It works great.
On the Pi-hole admin page, under Settings —> DNS. It has a list of built-in upstream DNS servers, e.g. Google, OpenDNS, Quad9, Cloudflare, etc.

  • OpenDNS: OpenDNS is my original first choice since I have been using its filter for a long time as I mentioned above. My only concern with OpenDNS is that Cisco acquired OpenDNS in 2015; and not sure whether this free service will continue to be maintained. But I don’t have other good options at the time.
  • Google DNS: Google DNS has a relatively fast response time. It doesn’t provide content filtering. Google’s ad tracking business makes me stay away from its products when I can. So I stop using Google DNS.
  • Cloudflare DNS: Cloudflare DNS has a fast response time. It doesn’t provide content filtering. After knowing Mozilla includes Cloudflare DoH in their newer version of Firefox, I trust Cloudflare’s privacy practice. Combining with Pi-hole other filtering feature, I switch to Cloudflare DNS as my upstream DNS servers until I learn about CleanBrowsing DNS.
  • CleanBrowsing DNS: CleanBrowsing DNS’s free filters (Security, Adult, and Family) are silmiar to OpenDNS, except the free plan doesn’t offer custom filters (e.g. gaming, grambling, etc). However, its free adult or family filter can filter the search engines (Google and Bing) result in safe mode, which I don’t see in other DNS filter. So CleanBrowsing DNS’s audlt filter is my choice for the Pi-hole upstream DNS at the home network. (PS. the family filter blocks Reddit site. I agree some contents on Reddit is not appropriate, but it also has many good technical disucssion.)
CleanBrowsing DNS setup on Pi-hole
  • http://<pihole.ip.address>/admin
  • Setting, DNS, Upstream DNS Servers
  • Custom 1 (IPv4): 185.228.168.10
  • Custom 2 (IPv4): 185.228.169.11

Set up Pi-hole on a Raspberry Pi

My first and main usage for a Raspberry Pi is to set up Pi-hole to block Ad for all devices on my home network.

Installation

The Pi-hole instalaltion is simple.

curl -sSL https://install.pi-hole.net | bash

Basic operation command

  • check status
    • sudo pihole status
  • check version
    • sudo pihole -v
  • update Pi-hole
    • sudo pihole -up
  • change Pi-hole admin password
    • sudo pihole -a -p
  • update Pi-hole filter list (gravity.list)
    • sudo pihole -g
  • restart Pi-hole
    • sudo pihole restartdns
  • list domains in whilelist
    • sudo pihole -w -l
  • add a domain to whitelist
    • sudo pihole -w <domain>
  • remove a domain from whitelist
    • sudo pihole -w -d <domain>
  • list domains in blacklist
    • sudo pihole -b -l

Once the pi-hole is up and running, we can change the the DNS setting of the router’s DHCP to the pi-hole IP address, or manaully set it to the devices with the static IP address.

Pi-hole comes the default ad block list. I will discuss the selection of Upstream DNS servers and add additional filter in the next post.

Set up a Raspberry Pi

I’m going to write a series of posts about Raspberry Pi and how I use it to secure and optimize my home network. This first post covers the basic setup procedure. Please refer to the official setup guide if you need the step-by-step instruction.

Hardware

  • Raspberry Pi 3 Model B. At the time of writing, the latest board is Raspberry Pi 4 Mode B, which is more powerful. But any version Raspberry Pi board should work.
  • MicroSD card (minimum 4GB)
  • Micro USB power supply (2.5A). For Raspberry Pi 4, you need the USB-C power supply
  • USB keyboard and mouse
  • Full-size HDMI cable to connect Raspberry Pi to a monitor or TV duing the initial setup. For Raspberry Pi 4, you need the Micro HDMI cable
  • Raspberry Pi case

Operating system

  • Raspbian -  the offical Raspberry Pi operating system. I want to keep the OS footprint small and I don’t plan to connect to a monitor other than the initial setup, so I select Raspbian Lite image.

Basic procedure

  1. Download Raspbian Lite image
  2. Download balenaEtcher
  3. Launch balenaEtcher and write the Raspbian image to the SD card
  4. Insert the SD card, connect the network cable, the keyboard & mouse, monitor and power supply to the board
  5. Power on and walk through the setup
  6. At the console prompt, log in with the default credential: pi / raspberry
  7. passwd
    • change the user pi’s password 
  8. sudo raspi-config
    • configure time zone, locale and keyboard layout
    • enable SSH under Interfacing options
  9. sudo nano /etc/dhcpcd.conf
    • configure a static IP address on eth0 
    • interface eth0
      static ip_address=x.x.x.x/24
      static routers=x.x.x.x
      static domain_name_servers=x.x.x.x
  10. sudo nano /etc/sysctl.conf
    • disable IPv6, by adding a line “net.ipv6.conf.all.disable_ipv6 = 1"
    • sudo sysctl -p
  11. sudo apt update && sudo apt upgrade -y
  12. sudo reboot
  13. after reboot, verify SSH login

Now the Raspberry Pi is fully functional. I’m going to use it to set up Pi-Hole to block Ad, DNS-Over-HTTPS (DoH) for privacy, CUPS to add AirPrint to an USB printer, and more. Stay tuned.

View and Modify Delete On Termination Setting on AWS EBS Volume

When launching an AWS EC2 instance, you can specify whether a EBS volume will be deleted when the instance is terminated. The setting is “Delete On Termination” and it’s on by default.

Once the instance is launched, viewing or modifying this “Delete On Termination” setting is not obvious. This post is to document how I do it.

View “Delete On Termination” Setting on AWS EBS Volume

There are two ways:
  • Use EC2 management console
    • Click on Instances and select the instance
    • Under Description tab,scroll down to “Block devices"
    • Click on the device name to show the “Delete on termination” setting on the pop-up window
  • Use AWS CLI
    • Get the instance ID
      • $ aws ec2 describe-instances --query 'Reservations[*].Instances[*].InstanceId' --output text
    • Query the setting
      • aws ec2 describe-instances --filter "Name=instance-id,Values=i-0d0a326ab6691cf24" --query 'Reservations[*].Instances[*].BlockDeviceMappings[*].Ebs.DeleteOnTermination'

Modify “Delete On Termination” Setting on AWS EBS Volume



I cannot find a way to modify this setting on EC2 management console. It looks AWS CLI is the only way. Here is the CLI.

aws ec2 modify-instance-attribute --instance-id i-0d0a326ab6691cf24 --block-device-mappings file://./change.DeleteOnTermination.json

The json file format is below. Update the DeviceName to match the EBS volume that you wan to modify
[
{
"DeviceName": "/dev/xvda",
"Ebs": {
"DeleteOnTermination": false
}
}
]
After executing the CLI, refresh the management console and click on the volume name under the instance description to verify the setting has been modified.


Use WinSCP to Transfer Files in vCSA 6.7

This is a quick update on my previous post “ Use WinSCP to Transfer Files in vCSA 6.5 ”. When I try the same SFTP server setting in vCSA 6.7...