elliotkillick / qvm-create-windows-qube Goto Github PK

It may be possible to generate the ISO image without copying all of the files to a temp directory. The genisoimage command can take multiple dirs/files to overlay in the final image.

Try something like this:

genisoimage -udf -b boot.bin -no-emul-boot -allow-limited-size -quiet -o "$final_iso" "$iso_mntpoint" Autounattend.xml boot.bin 

I enter the command: ./download-windows.sh win10x64

but then I get the error message: Microsoft does not allow automatic downloading of this Windows media, so please download it manually here: https://www.microsoft.com/en-us/software-download/windows10ISO (Don't forget to verify it afterwards)

then I tried to download the ISO from the official Microsoft site: https://www.microsoft.com/en-us/software-download/windows10ISO , but the download always stops.

is the problem with Microsoft or am I doing something wrong?

Can you please enable RDP in the answer file so that we can log in (over a remote desktop session from another qube).
Otherwise qubes issue #1585 applies QubesOS/qubes-issues#1585

https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-8.1-and-8/hh825710(v=win.10)?redirectedfrom=MSDN

Although there's a lot of overlap, perhaps we can mine a few things from there.

https://github.com/Disassembler0/Win10-Initial-Setup-Script

Hi,

The 'install.sh' script says to verify the PGP Signature, but it's not clear to me which file I'm verifying the PGP signature of, nor which key to verify it against. Sorry if this seems like an asinine issue.

It would be really nice if it were possible to select the storage pool to use for the Windows installation. Cloning to shift pools after installation is really cumbersome.

I recently discovered I have a windows-mgmt app VM lying around and wonder if this serves any purpose after the initial installation of the Windows qube.

Can the windows-mgmt app VM be deleted? It would be nice to find a hint about this somewhere near the explanation of what it does and why it's there.

Thanks for listening :)

https://github.com/tabit-pro/Qway-qubes-repo/tree/master/qubes-windows-tools
https://groups.google.com/g/qubes-devel/c/lreMRsAPjnI

when I enter the command qvm-run -p --filter-escape-chars --no-color-output <qube_script_is_located_on> "cat '/home/user/Downloads/install.sh'" > install.sh in dom0 to copy install.sh to dom0, it doesn't work.

This I find bad...

qvm-run -p "$resources_qube" "cd ${resources_dir%/*} && git clone https://github.com/elliotkillick/qvm-create-windows-qube"

echo -e "${BLUE}[i]${NC} Please check for a \"Good signature\" from GPG..." >&2
qvm-run -q "$resources_qube" "gpg --keyserver keys.openpgp.org --recv-keys 018FB9DE6DFA13FB18FB5552F9B90D44F83DD5F2"
qvm-run -p "$resources_qube" "cd '$resources_dir' && git verify-commit \$(git rev-list --max-parents=0 HEAD)"

...because it runs commands that require networking from dom0, relies on networking and successful gpg verification.

feature request: please design this script form the perspective of already being installed in Qubes dom0 without any networking/extra gpg verification required. Packaging, so this could be reviewed by @QubesOS and installed in dom0 using qubes-dom0-update.

Thank you very much for this project. I tried it with the current win10x64 and I'm currently stuck several times at the same step:
[i] Preparing Windows media for automatic installation...
[i] Starting creation of win10-orig_01
[i] Commencing first part of Windows installation process...
[i] Commencing second part of Windows installation process...
[i] Preparing Qubes Windows Tools for automatic installation...
[i] Installing Qubes Windows Tools...

During the windows tools installation the volume d: fails during formating with ntfs. By trying to format it manually (format d: /fs:ntfs /Q) the following message is displayed:
"QuickFormatting 2.0 GB
Starting offset of the thinly provisioned or DAX partition is not aligned to a cluster boundary. Partition is 512 bytes aligned. To format with specified cluster size, align the partition to 4 KB.
Format failed."

The installation of Windows Tools cannot be continued.

Seems to be related to this issue: QubesOS/qubes-issues#5768

Any ideas how to resolve this?

Hi,

I followed the instructions in the README, but at the end of running install.sh there was no 'qvm-create-windows-qube.sh' for me to run. The windows-mgmt qube was installed (though I'm still a bit unclear on its purpose, I did notice that you said it was used for creating Windows qubes).

[user@disp6703 ~]$ git clone [email protected]:elliotkillick/qvm-create-windows-qube
Cloning into 'qvm-create-windows-qube'...
The authenticity of host 'github.com (140.82.114.3)' can't be established.
RSA key fingerprint is SHA256:nThbg6kXUpJWGl7E1IGOCspRomTxdCARLviKw6E5SY8.
Are you sure you want to continue connecting (yes/no/[fingerprint])? y
Please type 'yes', 'no' or the fingerprint: yes
Warning: Permanently added 'github.com,140.82.114.3' (RSA) to the list of known hosts.
[email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
[user@disp6703 ~]$

Hi,
I was unable to connect a USB printer/scanner to Windows. This may be an issue/limitation with Qubes Windows Tools.

ping @ElliotKillick Win7/Win10 works good. But there is a problems with Server versions. QWT stuck, so no network on qube...any chance to fix?

Official requirements say 2GB.

And indeed with 1GB the installation fails: https://openqa.qubes-os.org/tests/37189/video?filename=video.ogv&t=77

Then, the VM shutdowns, but qvm-create-windows-qube doesn't know it failed. It tries to start it again, and waits for the shutdown idefinitely (no timeout at this step...).

PS Yes, I'm trying to use the tool for automatic tests :)

Following urls now are broken. I checked that rest worked well.

win7x64-ultimate.iso 
https://download.microsoft.com/download/5/1/9/5195A765-3A41-4A72-87D8-200D897CBE21/7601.24214.180801-1700.win7sp1_ldr_escrow_CLIENT_ULTIMATE_x64FRE_en-us.iso

win2008r2.iso
 https://download.microsoft.com/download/7/5/E/75EC4E54-5B02-42D6-8879-D8D3A25FBEF7/7601.17514.101119-1850_x64fre_server_eval_en-us-GRMSXEVAL_EN_DVD.iso

This problem is such a headache. I have tested and experienced it on Win10 enterprise, ltse and pro. The VMs simply refuse to maintain sync. The enterprise and LTSE VMs refuse to sync to any of the time servers. You can manually set the clock but after a reboot the clock loses sync it can be up to a few hours to a day. The PRO vm i just setup now will sync with timeserver and get the correct time and instantly reboot it and the clock is out of sync a few hours.

Desperately looking for some help diagnosing this issue.

Thanks

Known issue: https://social.technet.microsoft.com/Forums/en-US/87b81fa7-f1d8-412f-91ac-e53c0a2578f3/windows-10-enterprise-ltsc-evaluation-dead-on-arrival-quotproduct-key-is-blockedquot?forum=win10itprosetup

#upstream #wontfix

I followed the instructions up to step 4 for installation. Once I executed the step 4 command in dom0 chmod +x install.sh && ./install.sh I get stuck at a loop saying it has failed to download windows due to public key not matching the pinned one will retry in 10 seconds (loop$ Can anyone advise me in noob friendly way what possible cause of problem is and how I might fix it? I followed the instructions verbatim and I am running Qubes 4.0.3 with all available updates installed.

Thanks

https://groups.google.com/g/qubes-users/c/vXDnBvjr6GA?pli=1

Relevant terminal logs linked within. Any help in getting a windows10 qube working would be appreciated.

Problem seems to be that qvm-create-windows-qube in dom0 doesn't pop.

Hi,

Please see: Qubes-Community/Contents#146

Neither of these following commands work (am currently on the R4.1 Beta):

sudo qubes-dom0-update qubes-windows-tools
sudo qubes-dom0-update --enablerepo=qubes-dom0-current-testing qubes-windows-tools

I get the following output:

 No Match for argument qubes-windows-tools
Nothing to download
Fetching updates failed with code 1; press Enter to exit

As a result the qvm-create-windows-qube tool fails.

I'm not sure where I should start debugging this.

I followed the installation instructions and installed Windows 10 Pro. Everything went well. I also got the expected errors regarding Qubes Windows Utils, .NET etc. During the installation I could see the Windows 10 UI, could see the reboots. All good. Then the installation finished with a success message:

And now I have no UI anymore. I can start the Windows 10 VM. In the Qubes Domain Widget I see the VM running and consuming CPU. I can start programs from the start menu and see the hard drive LED blink and CPU usage go up for the Windows VM.

But I see no actual window to interact with. Nothing opens.

Does this point to a specific problem? How would I diagnose it? Any hints are appreciated. I just need a window to click...

After following the instructions in the repository, the script tries to download Windows 7 from official sources, but curl fails to do so because the SSL PubKey does not match the pinned one. It endlessly tries it again in a 10 second loop.

I've setup windows 7 and 10 both working all right, however I've noticed that windows 7 on a maximized window gets 3832x2077 resolution and when I go full screen it automatically goes to 3840x2160 which is my dom0 resolution.

On Windows 10 though it gets 2560x1600 on a maximized window and when I go full-screen it doesn't change like it does with Windows 7. This is sort of half the screen and a little ugly to work with.

How would one change the windows 10 resolution to at least 3832x2077 ?

I don't know how to copy and paste out of dom0 so I will just retype this manually

$ qvm-create-windows-qube -n sys-whonix -oyw -i win10x64-enterprise-eval.iso -a win10x64-enterprise-eval.xml win10-work
Preparing Windows media for automatic installation ...
Starting creation of win10-work ...
Commencing first part of Windows installation process ...
Commencing second part of Windows installation process ...
Preparing Qubes Windows Tools for automatic installation ...
[!] An unexpected error as occured! Exiting ...

Error message ask for minimum 10GB disk space.

Extending the private storage to 10GB works fine.

I got messages as"curl: (90) SSL: public key does not match pinned public key!"
my system version is qubesos_4.0.2_rc3
error log:

[i] Cloning qvm-create-windows-qube GitHub repository...
Cloning into 'qvm-create-windows-qube'...
[i] Please check for a good PGP signature (Verify it out-of-band if necessary)...
gpg: Signature made Sat Nov 30 02:48:44 2019 EST
gpg: using EDDSA key 018FB9DE6DFA13FB18FB5552F9B90D44F83DD5F2
gpg: Good signature from "Elliot Killick [email protected]" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
Primary key fingerprint: 018F B9DE 6DFA 13FB 18FB 5552 F9B9 0D44 F83D D5F2
[i] Downloading Windows 7 (Other versions of Windows can be downloaded later by using download-windows.sh)...
[i] Downloading Windows media from Microsoft servers...
[i] Downloading Windows 7...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- 0:00:05 --:--:-- 0
curl: (90) SSL: public key does not match pinned public key!

QWT install does not complete as .NET 3.5 needed to be installed from Sever Manager.
Windows Server 2016 throws a source error (0x800f081f) when attempting to do so. (likely because the airgap hasn't been removed yet)

Expected behavior: QWT and dependencies install successfully.

Commenting lines 302-304 (qvm-firewall rules to maintain airgap) to fix the issue, understanding that the VM may try phoning home to MSFT if this is done.

May wish to investigate building the CAB into the ISO and running the DSIM e.g. https://docs.microsoft.com/en-us/windows-hardware/manufacture/desktop/deploy-net-framework-35-by-using-deployment-image-servicing-and-management--dism

I noticed in the Qubes Manager that I'm unable to increase the maximum RAM for my Windows VM. Is this by design? Should we do adjustments via CLI?

Since Qubes 4.1 it's possible to use debian as default template for all qubes. The script assumes a fedora default template and fails if the management qube is created with a debian template.

I see two ways to solve this:
a) make having a fedora template a requisit and extend the script to find a suitable template (latest installed fedora template)
b) make the script compatible with debian as well as fedora, so it does not matter which template is default

Workaround:
For now, installing a fedora template and setting it as default for the first run of the script works fine.

Sorry, I will close this as solved as the issue was caused by me having an older version of QWT installed

Solved by installing latest QWT 4.1.67 from here : https://github.com/tabit-pro/qubes-windows-tools-cross/releases

You even get audio support !

=== FOR THE RECORD ONLY ===

I deployed a Win10 VM onto Qubes 4.1 using qvm-create-windows-qube (thanks for the excellent tool).

• The VM is deployed OK and is usable, BUT :

• 1/ qvm-create-windows-qube never end and stays frozen (until I [Ctrl]-C it).

This seems to be caused by the fact that qvm-create-windows-qube is waiting for qvm-sync-appmenus and looping endlessly waiting as qubes-rpc service does not work on the qube - trying to start it plainly crashes the qube.

• 2/ Deployed qube has a working network connection but cannot reach the outside world

This is caused by the fact that the qube's firewall is forbidding access.

$ qvm-firewall <qube_name> reset

...solves this and restores proper network connectivity

• 3/ Packages requested by the “-p firefox” etc. fail installing. I assume that this is because the VM has no working network in the first place...

• 4/ Also download-windows.sh seems unable to download the Win10 ISO directly - it spits a browser URL for manual download instead - and the Win10 ISOs downloaded from Microsoft have different names i.e. Win10_21H2_EnglishInternational_x64.iso and qvm-create-windows-qube doesn't have their SHA256SUM so they can't be verified.

Besides this, the requested VM is deployed, working and usable, so thanks !

Kind regards.

In following the instructions given in the main page of this site I completed the following instruction:

qvm-run -p --filter-escape-chars --no-color-output untrusted "cat '/home/install.sh'" > install.sh

When doing an ls in Dom0, I could see the file was copied, though when I went into the file all of the contents were gone. To view the contents of the file I used nano intall.sh. I then did stat install.sh for the file in Dom0 as well as the file in the untrusted VM. In Dom0 the file size is 0 whereas the file size in untrusted is 3772. I have a 1.4M print-screen (jpg) of this but cannot attach it to this message. I am getting an error message in the github website stating "Something went really wrong, and we cannot process that file"

Not sure if this is better reported on the community forums or elsewhere, but figured I'd start here.

QWT seems to need to create a "private volume" during the install process, and it is failing to format the disk (see screenshot). Going to D:\ in explorer confirms that the volume is un-formatted.

This is on Windows 10 Pro 20H2, build 19042.631.

It would be comfortable for additional automation tools if qvm-create-windows-qube allows --resources-qube.

Right now I create disposable vm as windows-mgmt. It have random name. But then I can't use qvm-create-windows-qube as it hardcoded to windows-mgmt

After "Air gapping windows-mgmt..." an error is thrown:
"qvm-prefs: error: no such property: 'netvm'

This error is misleading (see: QubesOS/qubes-issues#3528 ) - what is really happening is that the netvm cannot be detached while windows-mgmt is running.

Jan 13 23:40:52 dom0 libvirtd[2785]: 2020-01-14 04:40:52.357+0000: 2816: error : libxlDomainDetachNetDevice:3984 : internal error: libxenlight failed to detach network device
Jan 13 23:40:52 dom0 qubesd[2756]: unhandled exception while calling src=b'dom0' meth=b'admin.vm.property.Set' dest=b'windows-mgmt' arg=b'netvm' len(untrusted_payload)=0
Jan 13 23:40:52 dom0 qubesd[2756]: Traceback (most recent call last):
Jan 13 23:40:52 dom0 qubesd[2756]:   File "/usr/lib/python3.5/site-packages/qubes/api/__init__.py", line 275, in respond
Jan 13 23:40:52 dom0 qubesd[2756]:     untrusted_payload=untrusted_payload)
Jan 13 23:40:52 dom0 qubesd[2756]:   File "/usr/lib64/python3.5/asyncio/futures.py", line 381, in __iter__
Jan 13 23:40:52 dom0 qubesd[2756]:     yield self  # This tells Task to wait for completion.
Jan 13 23:40:52 dom0 qubesd[2756]:   File "/usr/lib64/python3.5/asyncio/tasks.py", line 310, in _wakeup
Jan 13 23:40:52 dom0 qubesd[2756]:     future.result()
Jan 13 23:40:52 dom0 qubesd[2756]:   File "/usr/lib64/python3.5/asyncio/futures.py", line 294, in result
Jan 13 23:40:52 dom0 qubesd[2756]:     raise self._exception
Jan 13 23:40:52 dom0 qubesd[2756]:   File "/usr/lib64/python3.5/asyncio/tasks.py", line 240, in _step
Jan 13 23:40:52 dom0 qubesd[2756]:     result = coro.send(None)
Jan 13 23:40:52 dom0 qubesd[2756]:   File "/usr/lib64/python3.5/asyncio/coroutines.py", line 210, in coro
Jan 13 23:40:52 dom0 qubesd[2756]:     res = func(*args, **kw)
Jan 13 23:40:52 dom0 qubesd[2756]:   File "/usr/lib/python3.5/site-packages/qubes/api/admin.py", line 244, in vm_property_set
Jan 13 23:40:52 dom0 qubesd[2756]:     untrusted_payload=untrusted_payload)
Jan 13 23:40:52 dom0 qubesd[2756]:   File "/usr/lib/python3.5/site-packages/qubes/api/admin.py", line 264, in _property_set
Jan 13 23:40:52 dom0 qubesd[2756]:     setattr(dest, self.arg, newvalue)
Jan 13 23:40:52 dom0 qubesd[2756]:   File "/usr/lib/python3.5/site-packages/qubes/vm/__init__.py", line 460, in __set__
Jan 13 23:40:52 dom0 qubesd[2756]:     super(VMProperty, self).__set__(instance, value)
Jan 13 23:40:52 dom0 qubesd[2756]:   File "/usr/lib/python3.5/site-packages/qubes/__init__.py", line 261, in __set__
Jan 13 23:40:52 dom0 qubesd[2756]:     name=self.__name__, newvalue=value, oldvalue=oldvalue)
Jan 13 23:40:52 dom0 qubesd[2756]:   File "/usr/lib/python3.5/site-packages/qubes/events.py", line 198, in fire_event
Jan 13 23:40:52 dom0 qubesd[2756]:     pre_event=pre_event)
Jan 13 23:40:52 dom0 qubesd[2756]:   File "/usr/lib/python3.5/site-packages/qubes/events.py", line 166, in _fire_event
Jan 13 23:40:52 dom0 qubesd[2756]:     effect = func(self, event, **kwargs)
Jan 13 23:40:52 dom0 qubesd[2756]:   File "/usr/lib/python3.5/site-packages/qubes/vm/mix/net.py", line 430, in on_property_pre_set_netvm
Jan 13 23:40:52 dom0 qubesd[2756]:     self.detach_network()
Jan 13 23:40:52 dom0 qubesd[2756]:   File "/usr/lib/python3.5/site-packages/qubes/vm/mix/net.py", line 338, in detach_network
Jan 13 23:40:52 dom0 qubesd[2756]:     vm=self))
Jan 13 23:40:52 dom0 qubesd[2756]:   File "/usr/lib/python3.5/site-packages/qubes/app.py", line 101, in wrapper
Jan 13 23:40:52 dom0 qubesd[2756]:     return attr(*args, **kwargs)
Jan 13 23:40:52 dom0 qubesd[2756]:   File "/usr/lib64/python3.5/site-packages/libvirt.py", line 1170, in detachDevice
Jan 13 23:40:52 dom0 qubesd[2756]:     if ret == -1: raise libvirtError ('virDomainDetachDevice() failed', dom=self)
Jan 13 23:40:52 dom0 qubesd[2756]: libvirt.libvirtError: internal error: libxenlight failed to detach network device

Proposed solution: issue shutdown command to windows-mgmt VM first, validate shutdown, then continue.

B

I execute the command:
./qvm-create-windows-qube.sh -n sys-firewall -oyp firefox,notepadplusplus,office365proplus -i win10x64.iso -a win10x64-pro.xml work-win10
but the following error message appears:
File not found in windows-mgmt:/home/user/Documents/qvm-create-windows-qube/windows-media/isos: win10x64.iso

Am I doing something wrong, or have I forgotten an intermediate step. am grateful for any help :-)

The uploaded photo of dom0 terminal show some SSL error and GPG issue.

When installation was tried there were some internet connection issues.

But trying again does not work, apparently because windows-mgmt already exists. Should I try to delete it and try again? Or what?
Many thanks

I've found this amazing project for Windows users: https://ameliorated.info/. There is all the sources at https://git.ameliorated.info/malte/scripts. Maybe that could help into having more minimal windows images?

https://github.com/tabit-pro/qwt-crossbuild/

This may help address the win10/windows server issues w/r/t volume setup necessary to support templates.

The documentation should include an information that the script will install packages in the fedora-30 template.
I like to keep my default qubes template as they are and use them as "baseline" templates for other templates to which I install template specific packages.
As far as I can see it will only install one package datefudge into the fedora-30 template and as such the process is reversible.
Otherwise great work!

After following the install instructions I get this error. No idea why. Can you help?

[klaus@dom0 ~]$ ./install.sh
[i] Installing Qubes Windows Tools...
Using sys-whonix as UpdateVM to download updates for Dom0; this may take some time...
Qubes OS Repository for Dom0 2.9 MB/s | 3.0 kB 00:00
No match for argument: qubes-windows-tools
Error: Unable to find a match: qubes-windows-tools
[!] Error installing Qubes Windows Tools! Exiting...
[klaus@dom0 ~]$ ./install.sh
[i] Installing Qubes Windows Tools...
Using sys-whonix as UpdateVM to download updates for Dom0; this may take some time...
Qubes OS Repository for Dom0 2.9 MB/s | 3.0 kB 00:00
No match for argument: qubes-windows-tools
Error: Unable to find a match: qubes-windows-tools
[!] Error installing Qubes Windows Tools! Exiting...

Hey, I was reading some of the code and wanted to say your comments and error handling are great.

Qubes R4.0 introduced Qubes Admin API, which allows a VM to manage other VMs on a Qubes system without assuming full privileges of Dom0.

All the qvm-* commands work the same, so perhaps the only thing really needed might be replacing qubes-dom0-update with directly downloading the QWT ISO.

Some reason curl --output "$out_file" --"tlsv$tls_version" --proto =https -- "$url" doesn't download iso for majority of installations. But win81x64-enterprise-eval and win2012r2-eval work well.
The wget works well with all installations.

scurl_file() {
    out_file="$1"
    tls_version="$2"
    url="$3"

    # Map the TLS version to the corresponding `--secure-protocol` option in wget
    case "$tls_version" in
        1.2)
            secure_protocol="TLSv1_2"
            ;;
        1.3)
            secure_protocol="TLSv1_3"
            ;;
        *)
            echo -e "${RED}[!]${NC} Unsupported TLS version: $tls_version. Exiting..." >&2
            exit 1
            ;;
    esac

    until wget --output-document="$out_file" --secure-protocol="$secure_protocol" --https-only "$url"; do
        echo -e "${RED}[!]${NC} Failed to download Windows! Is there an Internet connection? Retrying in 10 seconds..." >&2
        sleep 10
    done
}

Installation worked properly however usb is not working in the vm and following driver seems to have problems in the windows device manager :

• Universal Serial Bus (USB) Controller
• XP9001 XENBUS VBD
• XP0001 XENBUS VIF

I followed all instructions up to step 5. No errors in dom0. I gave the windows-mgmt vm network access (i accept risk) and then in terminal for that vm i typed windows-media/isos/download-windows.sh

Please help . I have been reading this over and over trying find what i missed and i cant figure out whats wrong

According to feedback from Qubes OS Forum it is critical to disable hibernation with Windows 10 at preparation stage. Looks like auto-qwt/run.bat is the place for it.

P.S. Very nice tool, thank you.

The documentation says "A more streamlined and secure installation process with packaging will be shipping with Qubes R4.1", but they have already released "release candidate 3" for 4.1 (4.1.0-rc3), which does not seem to include qvm-create-windows-qube. Is this sill intended to make it in before the 4.1 actual release? (would be nice)

Suggestion: Add instruction how to update script properly. You make some useful commits, the instruction required in readme about the proper way to Update the script (or uninstall and reinstall).