741
737
>>> LaunchpadZopelessLayer.switchDbUser(config.builddmaster.dbuser)
743
For building a candidate in the release pocket for the main component
744
and the primary archive It will pass an 'archives' argument to the
745
slave that contains sources.list entries for each pocket required in
746
the primary archive dependency tree.
748
We also pass arguments called 'suite' which is the current distroseries and
749
pocket, (e.g. edgy-updates) and 'archive_purpose' which contains the build's
750
archive.purpose (e.g. PRIMARY or PPA). These latter two arguments are
751
used in the chroot to determine whether it needs to turn on some features
752
or not (like pkgstriptranslations and pkgmaintainermangler).
754
Please note also that the 'archive_private' flag is passed to the slave
755
builder. It is True for private archives and False otherwise.
757
>>> a_builder.setSlaveForTesting(OkSlave())
758
>>> a_builder.is_available
760
>>> candidate = a_build.queueBuild()
761
>>> removeSecurityProxy(a_builder)._dispatchBuildCandidate(candidate)
762
ensurepresent called, url=...
763
ensurepresent called,
764
url=http://localhost:58000/3/firefox_0.9.2.orig.tar.gz
767
deb http://ftpmaster.internal/ubuntu hoary main
770
Archive Purpose: PRIMARY
771
Archive Private: False
773
>>> candidate.destroySelf()
775
Currently we can theoretically dispatch a build candidate for a
776
builder in 'manual' mode.
778
Although this will not be optimal, because we can only
779
do it once the manual builder has been collected (due to the
780
BuildQueue.builder constraint). Also because we don't yet provide a
781
API/UI method to request the dispatch in advance.
783
>>> a_builder.manual = True
785
>>> a_builder.setSlaveForTesting(OkSlave())
786
>>> a_builder.is_available
788
>>> candidate = a_build.queueBuild()
789
>>> removeSecurityProxy(a_builder)._dispatchBuildCandidate(candidate)
790
ensurepresent called, url=...
791
ensurepresent called,
792
url=http://localhost:58000/3/firefox_0.9.2.orig.tar.gz
795
deb http://ftpmaster.internal/ubuntu hoary main
798
Archive Purpose: PRIMARY
799
Archive Private: False
801
>>> candidate.destroySelf()
803
739
Partner archive builds will set up the 'archives' argument such that it
804
740
references all the required pockets/components in the primary archive, in
805
741
addition to a reference to the release pocket in the partner archive itself.
823
755
any PPA/suite will fail during the first 20 minutes because no empty
824
756
indexes are published.
826
Note that only a published binary in the right context will make the
827
archive relevant, anything PENDING or published in another context
830
>>> warty = getUtility(IDistributionSet)['ubuntu']['warty']
831
>>> create_binary_publication_for(
832
... partner_archive, warty, PackagePublishingStatus.PUBLISHED)
834
>>> hoary = getUtility(IDistributionSet)['ubuntu']['hoary']
835
>>> create_binary_publication_for(
836
... partner_archive, hoary, PackagePublishingStatus.PENDING)
838
So, at moment, partner archive is still not relevant for builds in
839
hoary/i386. It's not passed to the builder.
841
>>> removeSecurityProxy(a_builder)._dispatchBuildCandidate(candidate)
842
ensurepresent called, url=...
843
ensurepresent called,
844
url=http://localhost:58000/3/firefox_0.9.2.orig.tar.gz
847
deb http://ftpmaster.internal/ubuntu hoary main restricted universe multiverse
848
deb http://ftpmaster.internal/ubuntu hoary-security main restricted universe multiverse
849
deb http://ftpmaster.internal/ubuntu hoary-updates main restricted universe multiverse
852
Archive Purpose: PARTNER
853
Archive Private: False
857
>>> candidate.destroySelf()
858
>>> a_builder.setSlaveForTesting(OkSlave())
859
>>> a_builder.is_available
862
>>> candidate = a_build.queueBuild()
863
>>> setupBuildQueue(candidate, a_builder)
864
>>> last_stub_mail_count = len(stub.test_emails)
866
>>> removeSecurityProxy(a_build).archive = ubuntu.main_archive
867
>>> candidate.destroySelf()
869
But this time We will create a valid publication on partner hoary/i386.
871
>>> from lp.soyuz.interfaces.component import IComponentSet
873
>>> LaunchpadZopelessLayer.switchDbUser('launchpad')
874
>>> login('foo.bar@canonical.com')
875
>>> pub_source = test_publisher.getPubSource(
876
... archive=partner_archive, distroseries=hoary,
877
... status=PackagePublishingStatus.PUBLISHED,
878
... component='partner')
879
>>> pub_binaries = test_publisher.getPubBinaries(
880
... archive=partner_archive, pub_source=pub_source,
881
... distroseries=hoary, status=PackagePublishingStatus.PUBLISHED)
882
>>> partner_build = pub_binaries[0].binarypackagerelease.build
883
>>> partner_candidate = partner_build.buildqueue_record
885
>>> LaunchpadZopelessLayer.switchDbUser(config.builddmaster.dbuser)
887
Now when we dispatch the partner build, since it has one published
888
binary in hoary/i386, the partner archive gets included in the builder
891
>>> removeSecurityProxy(
892
... a_builder)._dispatchBuildCandidate(partner_candidate)
893
ensurepresent called, url=...
894
ensurepresent called, url=http://localhost:58000/.../foo_666.dsc
897
deb http://ftpmaster.internal/ubuntu hoary
898
main restricted universe multiverse
899
deb http://ftpmaster.internal/ubuntu hoary-security
900
main restricted universe multiverse
901
deb http://ftpmaster.internal/ubuntu hoary-updates
902
main restricted universe multiverse
903
deb http://ftpmaster.internal/ubuntu-partner hoary partner
905
Ogre-component: partner
906
Archive Purpose: PARTNER
907
Archive Private: False
909
>>> partner_candidate.destroySelf()
911
Similarly, PPA builds pass the 'archives' arguments:
913
>>> from canonical.launchpad.interfaces import IPersonSet
914
>>> cprov_archive = getUtility(IPersonSet).getByName('cprov').archive
915
>>> removeSecurityProxy(a_build).archive = cprov_archive
916
>>> a_builder.virtualized = True
917
>>> a_builder.vm_host = 'localhost.ppa'
919
>>> a_builder.setSlaveForTesting(OkSlave())
920
>>> a_builder.is_available
923
>>> candidate = a_build.queueBuild()
924
>>> setupBuildQueue(candidate, a_builder)
925
>>> last_stub_mail_count = len(stub.test_emails)
927
Exactly as Partner, Celso's PPA won't be included if it doesn't
928
contain any published binary in hoary/i386. We will create it before
931
>>> create_binary_publication_for(
932
... cprov_archive, hoary, PackagePublishingStatus.PUBLISHED)
934
>>> removeSecurityProxy(a_builder)._dispatchBuildCandidate(candidate)
935
ensurepresent called, url=...
936
ensurepresent called,
937
url=http://localhost:58000/3/firefox_0.9.2.orig.tar.gz
940
deb http://ftpmaster.internal/ubuntu hoary main restricted universe multiverse
941
deb http://ftpmaster.internal/ubuntu hoary-security main restricted universe multiverse
942
deb http://ftpmaster.internal/ubuntu hoary-updates main restricted universe multiverse
943
deb http://ppa.launchpad.dev/cprov/ppa/ubuntu hoary main
947
Archive Private: False
949
If the build is for a private PPA, the slave scanner will pass a
950
sources.list entry that contains a password to access the archive.
952
>>> from canonical.testing import LaunchpadZopelessLayer
954
>>> LaunchpadZopelessLayer.switchDbUser('launchpad')
955
>>> login('foo.bar@canonical.com')
956
>>> build = getUtility(IBinaryPackageBuildSet).getByQueueEntry(
958
>>> for build_file in build.source_package_release.files:
959
... removeSecurityProxy(build_file).libraryfile.restricted = True
960
>>> private_ppa = factory.makeArchive(
961
... owner=cprov_archive.owner, name='pppa', private=True,
962
... virtualized=False, distribution=ubuntu)
964
It's necessary to publish some binaries into the private PPA, otherwise
965
the PPA won't be included as a dependency in the sources list below.
967
>>> binaries = test_publisher.getPubBinaries(
968
... distroseries=ubuntu['hoary'], archive=private_ppa,
969
... status=PackagePublishingStatus.PUBLISHED)
970
>>> removeSecurityProxy(build).archive = private_ppa
972
>>> LaunchpadZopelessLayer.switchDbUser(test_dbuser)
975
Dispatch the build again. Celso's archive sources.list entry now has the
976
buildd:secret@ part in the URL.
978
Also note that when ensurepresent() is called, it receives a URL that
979
points to the private archive rather than the librarian for the private
980
firefox file. This is because the build slaves are not allowed to
981
access the restricted librarian as it cannot provide access via
982
credentials, unlike the archive itself.
984
Finally, the archive purpose is overridden to PRIMARY instead of PPA
985
for any archives that have require_virtualized as False.
987
In this circumstance, it also uses the component override from the PRIMARY
988
archive and not the one from the PPA, which on the absence of ancestry
989
defaults to 'universe'.
991
>>> build = getUtility(IBinaryPackageBuildSet).getByQueueEntry(candidate)
992
>>> print build.current_component.name
995
This is so that the mangling tools will run over the built packages.
997
>>> removeSecurityProxy(a_builder)._dispatchBuildCandidate(candidate)
998
ensurepresent called, url=...
999
ensurepresent called,
1000
url=http://private-ppa.../cprov/pppa/.../firefox_0.9.2.orig.tar.gz
1001
URL authorisation with buildd/sekrit
1004
deb http://buildd:sekrit@private-ppa.../cprov/pppa/ubuntu hoary main
1005
deb http://ftpmaster.internal/ubuntu hoary
1006
main restricted universe multiverse
1007
deb http://ftpmaster.internal/ubuntu hoary-security
1008
main restricted universe multiverse
1009
deb http://ftpmaster.internal/ubuntu hoary-updates
1010
main restricted universe multiverse
1012
Ogre-component: universe
1013
Archive Purpose: PRIMARY
1014
Archive Private: True
1016
We will create an ancestry in the primary archive target to the 'main'
1017
component and this time the dispatching will follow that component.
1019
>>> sourcename = build.source_package_release.name
1021
>>> LaunchpadZopelessLayer.switchDbUser('launchpad')
1022
>>> login('foo.bar@canonical.com')
1024
>>> ancestry = test_publisher.getPubSource(
1025
... sourcename=sourcename, version='0.1', distroseries=hoary)
1027
>>> print ancestry.displayname
1028
mozilla-firefox 0.1 in hoary
1030
>>> print ancestry.component.name
1034
>>> LaunchpadZopelessLayer.switchDbUser(config.builddmaster.dbuser)
1035
>>> login(ANONYMOUS)
1037
>>> removeSecurityProxy(a_builder)._dispatchBuildCandidate(candidate)
1038
ensurepresent called, ...
1040
Ogre-component: main
1043
>>> candidate.destroySelf()
1045
758
Since this is a build in a private archive, the log was uploaded to
1046
759
the restricted librarian.
1048
>>> candidate = a_build.queueBuild()
761
>>> removeSecurityProxy(build).archive = private_ppa
763
>>> candidate = build.queueBuild()
1049
764
>>> setupBuildQueue(candidate, a_builder)
1050
765
>>> build.upload_log = None
1051
766
>>> candidate.builder.setSlaveForTesting(WaitingSlave('BuildStatus.OK'))
1088
803
>>> LaunchpadZopelessLayer.switchDbUser('launchpad')
1089
804
>>> login('foo.bar@canonical.com')
1091
We'll switch the build's archive back to Celso's PPA and set the PPA to
1092
virtualized before adding the dependency on Mark's PPA.
1094
>>> removeSecurityProxy(build).archive = cprov_archive
1095
>>> cprov_archive.require_virtualized = True
1096
>>> for build_file in a_build.source_package_release.files:
1097
... removeSecurityProxy(build_file).libraryfile.restricted = False
1098
>>> mark_archive = getUtility(IPersonSet).getByName('mark').archive
1100
>>> unused_dep = cprov_archive.addArchiveDependency(
1101
... mark_archive, PackagePublishingPocket.RELEASE,
1102
... getUtility(IComponentSet)['main'])
1105
>>> LaunchpadZopelessLayer.switchDbUser(test_dbuser)
1106
>>> login(ANONYMOUS)
1108
Now we can see that a build from Celso's PPA will be able to install
1109
dependencies from Mark's PPA, if Mark's PPA has at least one binary
1110
published in hoary/i386, which is not the case.
1112
>>> a_builder.setSlaveForTesting(OkSlave())
1113
>>> a_builder.is_available
1116
>>> candidate = a_build.queueBuild()
1117
>>> setupBuildQueue(candidate, a_builder)
1118
>>> last_stub_mail_count = len(stub.test_emails)
1120
>>> removeSecurityProxy(a_builder)._dispatchBuildCandidate(candidate)
1121
ensurepresent called, url=...
1122
ensurepresent called,
1123
url=http://localhost:58000/3/firefox_0.9.2.orig.tar.gz
1126
deb http://ftpmaster.internal/ubuntu hoary main restricted universe multiverse
1127
deb http://ftpmaster.internal/ubuntu hoary-security main restricted universe multiverse
1128
deb http://ftpmaster.internal/ubuntu hoary-updates main restricted universe multiverse
1129
deb http://ppa.launchpad.dev/cprov/ppa/ubuntu hoary main
1131
Ogre-component: main
1132
Archive Purpose: PPA
1133
Archive Private: False
1135
We will create the required publication in Mark's PPA and try again.
1137
>>> candidate.destroySelf()
1138
>>> a_builder.setSlaveForTesting(OkSlave())
1139
>>> a_builder.is_available
1142
>>> candidate = a_build.queueBuild()
1143
>>> setupBuildQueue(candidate, a_builder)
1144
>>> last_stub_mail_count = len(stub.test_emails)
1146
>>> create_binary_publication_for(
1147
... mark_archive, hoary, PackagePublishingStatus.PUBLISHED)
1149
>>> removeSecurityProxy(a_builder)._dispatchBuildCandidate(candidate)
1150
ensurepresent called, url=...
1151
ensurepresent called,
1152
url=http://localhost:58000/3/firefox_0.9.2.orig.tar.gz
1155
deb http://ftpmaster.internal/ubuntu hoary main restricted universe multiverse
1156
deb http://ftpmaster.internal/ubuntu hoary-security main restricted universe multiverse
1157
deb http://ftpmaster.internal/ubuntu hoary-updates main restricted universe multiverse
1158
deb http://ppa.launchpad.dev/cprov/ppa/ubuntu hoary main
1159
deb http://ppa.launchpad.dev/mark/ppa/ubuntu hoary main
1161
Ogre-component: main
1162
Archive Purpose: PPA
1163
Archive Private: False
1165
807
Clean up before continuing:
1167
>>> candidate.destroySelf()
1168
809
>>> a_builder.virtualized = False
1169
810
>>> removeSecurityProxy(a_build).archive = ubuntu.main_archive
1172
Builddmaster stops before starting to build a denied build.
1173
Since hoary is in development, we are not able to dispatch
1174
builds for post-release pockets:
1176
>>> candidate = a_build.queueBuild()
1177
>>> setupBuildQueue(candidate, a_builder)
1178
>>> last_stub_mail_count = len(stub.test_emails)
1180
Make a build in the updates pocket:
1182
>>> hoary = hoary_i386.distroseries
1183
>>> hoary_evo = hoary.getSourcePackage(
1184
... 'evolution').currentrelease.sourcepackagerelease
1185
>>> updates_build = hoary_evo.createBuild(
1186
... distro_arch_series=hoary_i386,
1187
... pocket=PackagePublishingPocket.UPDATES,
1188
... processor=hoary_i386.default_processor,
1189
... archive=hoary_i386.main_archive)
1190
>>> updates_bqItem = updates_build.queueBuild()
1192
>>> hoary_i386.distroseries.status.name
1194
>>> removeSecurityProxy(a_builder)._dispatchBuildCandidate(updates_bqItem)
1195
Traceback (most recent call last):
1197
AssertionError: i386 build of evolution 1.0 in ubuntu hoary UPDATES (...) can not be built for pocket UPDATES: invalid pocket due to the series status of hoary.
1199
== Pocket dependencies ==
1201
Change the distroseries status for testing. FROZEN allows building in
1204
>>> from canonical.launchpad.interfaces import SeriesStatus
1205
>>> hoary_i386.distroseries.status = SeriesStatus.FROZEN
1207
Now we can start a build in other pockets, and see what archives are
1208
passed to the slave.
1210
A build in the updates pocket:
1212
>>> a_builder.currentjob.destroySelf()
1214
>>> bqItem3 = a_build.queueBuild()
1215
>>> build = getUtility(IBinaryPackageBuildSet).getByQueueEntry(bqItem3)
1216
>>> removeSecurityProxy(build).pocket = (
1217
... PackagePublishingPocket.UPDATES)
1218
>>> last_stub_mail_count = len(stub.test_emails)
1219
>>> removeSecurityProxy(a_builder)._dispatchBuildCandidate(bqItem3)
1220
ensurepresent called, url=...
1221
ensurepresent called,
1222
url=http://localhost:58000/3/firefox_0.9.2.orig.tar.gz
1225
deb http://ftpmaster.internal/ubuntu hoary main
1226
deb http://ftpmaster.internal/ubuntu hoary-security main
1227
deb http://ftpmaster.internal/ubuntu hoary-updates main
1228
Suite: hoary-updates
1229
Ogre-component: main
1230
Archive Purpose: PRIMARY
1231
Archive Private: False
1233
A build in the proposed pocket:
1235
>>> a_builder.currentjob.destroySelf()
1237
>>> bqItem3 = a_build.queueBuild()
1238
>>> removeSecurityProxy(build).pocket = (
1239
... PackagePublishingPocket.PROPOSED)
1240
>>> last_stub_mail_count = len(stub.test_emails)
1241
>>> removeSecurityProxy(a_builder)._dispatchBuildCandidate(bqItem3)
1242
ensurepresent called, url=...
1243
ensurepresent called,
1244
url=http://localhost:58000/3/firefox_0.9.2.orig.tar.gz
1247
deb http://ftpmaster.internal/ubuntu hoary main
1248
deb http://ftpmaster.internal/ubuntu hoary-proposed main
1249
deb http://ftpmaster.internal/ubuntu hoary-security main
1250
deb http://ftpmaster.internal/ubuntu hoary-updates main
1251
Suite: hoary-proposed
1252
Ogre-component: main
1253
Archive Purpose: PRIMARY
1254
Archive Private: False
1256
A build in the backports pocket:
1258
>>> a_builder.currentjob.destroySelf()
1260
>>> bqItem3 = a_build.queueBuild()
1261
>>> removeSecurityProxy(build).pocket = (
1262
... PackagePublishingPocket.BACKPORTS)
1263
>>> last_stub_mail_count = len(stub.test_emails)
1264
>>> removeSecurityProxy(a_builder)._dispatchBuildCandidate(bqItem3)
1265
ensurepresent called, url=...
1266
ensurepresent called,
1267
url=http://localhost:58000/3/firefox_0.9.2.orig.tar.gz
1270
deb http://ftpmaster.internal/ubuntu hoary main restricted universe multiverse
1271
deb http://ftpmaster.internal/ubuntu hoary-backports main restricted universe multiverse
1272
deb http://ftpmaster.internal/ubuntu hoary-security main restricted universe multiverse
1273
deb http://ftpmaster.internal/ubuntu hoary-updates main restricted universe multiverse
1274
Suite: hoary-backports
1275
Ogre-component: main
1276
Archive Purpose: PRIMARY
1277
Archive Private: False
1279
A build in the security pocket:
1281
>>> a_builder.currentjob.destroySelf()
1283
>>> bqItem3 = a_build.queueBuild()
1284
>>> removeSecurityProxy(build).status = (
1285
... BuildStatus.NEEDSBUILD)
1286
>>> removeSecurityProxy(build).pocket = (
1287
... PackagePublishingPocket.SECURITY)
1288
>>> last_stub_mail_count = len(stub.test_emails)
1290
The pocket-dependency infrastructure is ready to deal with SECURITY
1291
pocket, however we explicitly skip security builds when dispatching
1292
because Embargoed-Archives and Restricted-UI implementations are not
1295
>>> removeSecurityProxy(a_builder)._dispatchBuildCandidate(bqItem3)
1296
Traceback (most recent call last):
1298
AssertionError: Soyuz is not yet capable of building SECURITY uploads.
1300
Builds for security pocket are marked as FAILEDTOBUILD inside the
1301
_findBuildCandidate() method, see doc/buildd-dispatching.txt
1304
814
== Builder Status Handler ==