Windows Taskcluster

buildprops.json

{
    "properties": {
        "buildnumber": 6573, 
        "product": "firefox", 
        "script_repo_revision": "production", 
        "branch": "try", 
        "repository": "", 
        "buildername": "WINNT 5.2 try build", 
        "basedir": "/c/builds/moz2_slave/try-w32-0000000000000000000000", 
        "project": "", 
        "platform": "win32", 
        "master": "http://buildbot-master87.bb.releng.scl3.mozilla.com:8101/", 
        "commit_titles": [
            "try: -b do -p all -u mochitest-o -t none"
        ], 
        "scheduler": "try-firefox", 
        "repo_path": "try", 
        "slavename": "b-2008-ix-0062", 
        "builduid": "a55c9943a9ba404c985f823c90242ee4", 
        "revision": "b01bbc3bb8ddb47a4c1523805a9ed5b4b693757c"
    }, 
    "sourcestamp": {
        "repository": "", 
        "hasPatch": false, 
        "project": "", 
        "branch": "try", 
        "changes": [
            {
                "category": null, 
                "files": [
                    {
                        "url": null, 
                        "name": "accessible/base/nsAccessibilityService.cpp"
                    }, 
                    {
                        "url": null, 
                        "name": "accessible/tests/mochitest/treeupdate/test_textleaf.html"
                    }
                ], 
                "repository": "", 
                "rev": "b01bbc3bb8ddb47a4c1523805a9ed5b4b693757c", 
                "who": "[email protected]", 
                "when": 1438714337, 
                "number": 6182936, 
                "comments": "try: -b do -p all -u mochitest-o -t none", 
                "project": "", 
                "at": "Tue 04 Aug 2015 11:52:17", 
                "branch": "try", 
                "revlink": "https://hg.mozilla.org/try/rev/b01bbc3bb8ddb47a4c1523805a9ed5b4b693757c", 
                "properties": [
                    [
                        "commit_titles", 
                        [
                            "try: -b do -p all -u mochitest-o -t none"
                        ], 
                        "BaseHgPoller"
                    ]
                ], 
                "revision": "b01bbc3bb8ddb47a4c1523805a9ed5b4b693757c"
            }
        ], 
        "revision": "b01bbc3bb8ddb47a4c1523805a9ed5b4b693757c"
    }
}

steps.md

# Install chocolatey
c:/> @powershell -NoProfile -ExecutionPolicy Bypass -Command "iex ((new-object net.webclient).DownloadString('https://chocolatey.org/install.ps1'))" && SET PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin

choco install visualstudiocommunity2013
choco install vcredist2008 

$client = new-object net.webclient

# Enable dotnet 3.5
# https://technet.microsoft.com/en-us/library/Dn482071.aspx
# install directx sdk
$client.DownloadFile("http://download.microsoft.com/download/A/E/7/AE743F1F-632B-4809-87A9-AA1BB3458E31/DXSDK_Jun10.exe", "DXSDK_Jun10.exe")
Start-Process "DXSDK_Jun10.exe" -ArgumentList "/silent" 

# Install mozillabuild
$client.DownloadFile("https://ftp.mozilla.org/pub/mozilla.org/mozilla/libraries/win32/MozillaBuildSetup-Latest.exe", "MozillaBuildSetup-Latest.exe")
Start-Process "MozillaBuildSetup-Latest.exe"

#install dependenencies for source
choco install nodejs.install 
npm install -g [email protected]

Notes

• Following both the Win64 Machine Setup on the wiki, and the build prerequisites on MDN.

  I figure that we don't need some of the packages on the first image, so I'm installing dependencies as I go. ffledgling gave me access to one of his images, but I may or may not use it pending on the difficulty of installing the current dependencies. It seems managable so far. I may decide to follow mrrrgn's linux64 script, which seems to be relatively clean. If mozharness ends up being cross platform in terms of configuration, I can probably reuse a lot of it.

  [Mozilla-Central] /testing/docker/desktop-build

• Running on an m3.large instance. This will probably change to a m3.xlarge instance once I have all the setup done.

  jonasfj> amiyaguchi, for development just launch one... we like c3.xlarge or c3.2xlarge for building... just use whatever you think you'll use in production...

• Ran out of space on the m3.large instance, which is limited to 35GBs. Followed the directions on how to expand the ebs-volume

windows_cmd.md

./mozharness/scripts/fx_desktop_build.py `
--config builds/releng_base_windows_32_builds.py `
--disable-mock `
--no-setup-mock `
--no-clone-tools `
--no-clobber `
--no-update `
--no-upload-files `
--no-sendchange `
--log-level=debug `
--work-dir=C:\build `
--no-action=generate-build-stats

./mozharness/scripts/fx_desktop_build.py \
--config builds/releng_base_windows_32_builds.py \
--disable-mock \
--no-setup-mock \
--no-clone-tools \
--no-clobber \
--no-update \
--no-upload-files \
--no-sendchange \
--log-level=debug \
--work-dir=build \
--no-action=generate-build-stats \
--branch=try \
--build-pool=taskcluster

C:\mozilla-build\msys\bin\bash --login -c "python /c/Users/cltbld/sources/mozharness/scripts/fx_desktop_build.py --config builds/releng_base_windows_32_builds.py --disable-mock --no-setup-mock --no-update --no-upload-files --log-level=debug --branch=try --build-pool=taskcluster"

python mozharness/scripts/fx_desktop_build.py --config builds/releng_base_windows_32_builds.py --disable-mock --no-setup-mock --no-update --no-upload-files --no-clobber --no-checkout-sources --log-level=debug --branch=try --build-pool=taskcluster

Where are the buildprops.json made? I need a way to create them before they are made so I can actually do stuff.

You must run the mozharness script inside of a bash environment that has been started from a cmd, not powershell. Otherwiset the path environmental variable for %system32% doesn't get replaced.

eyJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJyYTIiLCJqdGkiOiJ0NjIiLCJ0eXAiOiJ1c3IifQ.ILHNmWvSVN93bd7ioPd_y71Tyn0-4MjJCtMs3laa6DU

https://queue.taskcluster.net/v1/task/eh5dnf7dQWuJEv6SP0bZ8w/artifacts/public/hg.mozilla.org/mozilla-central.tar.gz

• Download buildbotve folder and install into mozilla-build
• Install pywin32 for 32bit windows
• Download tooltool
• make clean
• make

22:22:09 INFO - Copy/paste: c:\Users\Administrator\sources\build\venv\Scripts\pip install --download-cache c:\Users\Administrator\sources\build\venv\cache --timeout 120 requests==2.2.1

22:22:10 ERROR - Traceback (most recent call last):
22:22:10 INFO - File "c:\Users\Administrator\sources\build\venv\Scripts\pip-script.py", line 9, in
22:22:10 INFO - load_entry_point('pip==0.8.2', 'console_scripts', 'pip')()
22:22:10 INFO - File "c:\Users\Administrator\sources\build\venv\lib\site-packages\distribute-0.6.14-py2.7.egg\pkg
_resources.py", line 305, in load_entry_point
22:22:10 INFO - return get_distribution(dist).load_entry_point(group, name)
22:22:10 INFO - File "c:\Users\Administrator\sources\build\venv\lib\site-packages\distribute-0.6.14-py2.7.egg\pkg
_resources.py", line 2244, in load_entry_point
22:22:10 INFO - return ep.load()
22:22:10 INFO - File "c:\Users\Administrator\sources\build\venv\lib\site-packages\distribute-0.6.14-py2.7.egg\pkg
resources.py", line 1954, in load
22:22:10 INFO - entry = import(self.module_name, globals(),globals(), ['name'])
22:22:10 INFO - File "c:\Users\Administrator\sources\build\venv\lib\site-packages\pip-0.8.2-py2.7.egg\pip__init
_.py", line 10, in
22:22:10 INFO - from pip.basecommand import command_dict, load_command, load_all_commands, command_names
22:22:10 INFO - File "c:\Users\Administrator\sources\build\venv\lib\site-packages\pip-0.8.2-py2.7.egg\pip\basecom
mand.py", line 6, in
22:22:10 INFO - import socket
22:22:10 INFO - File "c:\mozilla-build\python\Lib\socket.py", line 47, in
22:22:10 INFO - import _socket
22:22:10 INFO - ImportError: DLL load failed: The specified procedure could not be found.
22:22:10 WARNING - Return code: 1

The windows 2008 server edition serves its own version of python27.dll that gets served in the registry somewhere. It needs to be removed before the install will work properly.

General

• What does the buildbot master set up?
• I've heard that the build infrastructure has quite the history behind it. Has mozharness depreciated much of the buildbot functionality? What does buildbot still do, and what dependencies are left behind.
• How are the mercurial repositories set up? How are they updated from run to run if the slaves are cleaned up every time.

Specific

• Can you explain buildprops.json: Its generated by the buildbot master?
• What is the buildbotve, and why is it necessary.
• Is there a specific version of mozilla-build that is running on the slave images at the moment? Why is there such a difference between lastest mozilla build and the 2008 server image?

https://api.pub.build.mozilla.org/

• mvsc 2013
• i10n make
• missing seperator

https://pastebin.mozilla.org/8843093

export SCCACHE_DIR to use local directory, which doesn't use boto for uploads. See https://bugzilla.mozilla.org/show_bug.cgi?id=1000718

A minor update with progress on this issue.

With the image I got from the relops (spot-y-2008-2015-08-13-08-57, a try image), I've managed to get closer to a working builder. Currently, I am still unable to build anything successfully.

I'll outline a few of the work arounds I've had to make in order to get a semi working build.

• Update mozilla-build from 1.9 to 1.10
  This resolves a puzzling issue where mach configure (and in turn mach build) fails to read the nightly mozconfig pulled from browser/config/mozconfigs/win32/nightly during a mozharness build. The error looks like the following when run with start-shell-msvc-2013.bat.

  0:07.47 client.mk:117: *** missing separator (did you mean TAB instead of 8 spaces?). Stop.

When started with start-shell-l10n.bat, configuration succeeds, but compilation fails since none of the compilers have been added to the path.

• Disable remote sscache (to a certain extent)
  This involves setting SCCACHE_DIR= to a local directory, otherwise boto will fail due to authorization errors. This is probably due to the lack of a .boto with valid keys, which can otherwise be solved later.