Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

clone_items fails to clone referenced feature layer #2151

Open
rkraujutis opened this issue Oct 30, 2024 · 0 comments
Open

clone_items fails to clone referenced feature layer #2151

rkraujutis opened this issue Oct 30, 2024 · 0 comments
Labels
bug

Comments

@rkraujutis
Copy link

Describe the bug

Cloning feature layer item from one ArcGIS Online account to another ArcGIS Online account fails with error:

arcgis._impl.common._clone._ItemCreateException: ('Failed to create Feature Service 16.2.2e. Asmenų, ikiteisminiu tyrimu pripažintų nukentėjusiais nuo prekybos žmonėmis priverstinėms vedyboms (LT): Could not locate the Item: None', None)

To Reproduce
Steps to reproduce the behavior:

        source_content:ContentManager = source_gis.content
        source_item = source_content.get("2ff85137a9684a3bb6bdfc206cae2162")

        target_content:ContentManager = target_gis.content

        cloned_items:list[Item] = []
        try:
            cloned_items = target_content.clone_items([
                    source_item
                ],
                search_existing_items = False,
                copy_data = False,
            )
            for cloned_item in cloned_items:
                assert cloned_item.url == source_item.url, f"Expected that cloned item still references {source_item.url} but got {cloned_item.url}"
        except:
            logger.warning(f"Failed clone {source_item.itemid}", exc_info=True)

error:

Traceback (most recent call last):
  File "/opt/conda/lib/python3.12/site-packages/arcgis/gis/_impl/_content_manager/folder/core.py", line 583, in _process_item_status
    item = _arcgis_gis.Item(gis=self._gis, itemid=status_msg["itemId"])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/arcgis/gis/__init__.py", line 13214, in __init__
    if self._has_layers():
       ^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/arcgis/gis/__init__.py", line 13329, in _has_layers
    self.type == "Feature Collection"
    ^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/arcgis/gis/__init__.py", line 13496, in __getattr__
    self._hydrate()
  File "/opt/conda/lib/python3.12/site-packages/arcgis/gis/__init__.py", line 13448, in _hydrate
    itemdict = self._portal.get_item(self.itemid)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/arcgis/gis/_impl/_portalpy.py", line 1438, in get_item
    return self.con.post("content/items/" + itemid, self._postdata())
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/arcgis/gis/_impl/_con/_connection.py", line 1504, in post
    return self._handle_response(
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/arcgis/gis/_impl/_con/_connection.py", line 979, in _handle_response
    self._handle_json_error(data["error"], errorcode)
  File "/opt/conda/lib/python3.12/site-packages/arcgis/gis/_impl/_con/_connection.py", line 1002, in _handle_json_error
    raise Exception(errormessage)
Exception: Item does not exist or is inaccessible.
(Error Code: 400)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/lib/python3.12/site-packages/arcgis/_impl/common/_clone.py", line 2522, in clone
    new_item = self._add_new_item(item_properties)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/arcgis/_impl/common/_clone.py", line 2181, in _add_new_item
    new_item = job.result()
               ^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/opt/conda/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/arcgis/gis/_impl/_content_manager/folder/core.py", line 607, in _add_async_text
    return self._process_item_status(itemid=itemid)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/arcgis/gis/_impl/_content_manager/folder/core.py", line 588, in _process_item_status
    raise FolderException(f"Could not locate the Item: {itemid}")
arcgis.gis._impl._content_manager.folder._exceptions.FolderException: Could not locate the Item: None

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "REDACTED.py", line 27, in test_2ff85137a9684a3bb6bdfc206cae2162
    cloned_items = target_content.clone_items([
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/arcgis/gis/__init__.py", line 8757, in clone_items
    return deep_cloner.clone()
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/arcgis/_impl/common/_clone.py", line 1347, in clone
    results = executor.submit(self._clone, executor).result()
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/concurrent/futures/_base.py", line 456, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/opt/conda/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/arcgis/_impl/common/_clone.py", line 1321, in _clone
    raise ex
  File "/opt/conda/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/arcgis/_impl/common/_clone.py", line 2535, in clone
    raise _ItemCreateException(
arcgis._impl.common._clone

I see request:

POST https://REDACTED.maps.arcgis.com/sharing/rest/content/users/REDACTED/addItem HTTP/1.1
REDACTED

--b7d72677812d7812bcc692db225c8065
Content-Disposition: form-data; name="f"

json
--b7d72677812d7812bcc692db225c8065
Content-Disposition: form-data; name="async"

false
--b7d72677812d7812bcc692db225c8065
Content-Disposition: form-data; name="title"

16.2.2e. Asmenų, ikiteisminiu tyrimu pripažintų nukentėjusiais nuo prekybos žmonėmis priverstinėms vedyboms (LT)
--b7d72677812d7812bcc692db225c8065
Content-Disposition: form-data; name="type"

Feature Service
--b7d72677812d7812bcc692db225c8065
Content-Disposition: form-data; name="description"

<div><span style='font-size:small;'>LT</span><span style='color:rgb(0, 104, 157);'><font size='2'><b><span style='color:rgb(0, 104, 157);'><br /></span></b></font></span></div><span style='color:rgb(0, 104, 157);'><font size='2'><b><span style='color:rgb(0, 104, 157);'>16 TIKSLAS. Skatinti taikias ir įtraukias visuomenes darniam vystymuisi, suteikti visiems galimybes reikalauti teisingumo ir kurti veiksmingas, atsakingas ir įtraukias institucijas visais lygiais</span>:</b></font></span><div><font size='2'>Veiksmingai įgyvendintos žmogaus teisės ir laisvės padeda užtikrinti demokratines vertybes, stiprina saugumą, demokratinę visuomenę, mažina nusikalstamumą ir diskriminaciją. Įgyvendinant šį tikslą būtina suteikti galimybę visiems apginti savo teises teisėtomis priemonėmis, užtikrinti skaidrią ir efektyvią institucijų veiklą. Lietuvoje siekiama kurti saugią gyvenamąją aplinką, stiprinti kovą su sunkiais ir organizuotų nusikalstamų grupių daromais nusikaltimais, užtikrinti kvalifikuotą pagalbą nukentėjusiesiems.</font><div><font size='2'><font color='#ff0000'><b><span style='color:rgb(0, 104, 157);'>16.2. Uždavinys:</span> </b></font>Panaikinti vaikų paniekinimą, išnaudojimą, neteisėtą prekybą ir visų formų smurtą prieš vaikus bei jų kankinimą.</font></div><div><font size='2'><font color='#ff0000'><b><span style='color:rgb(0, 104, 157);'>16.2.2. JT rodiklis:</span> </b></font>Prekybos žmonėmis aukų skaičius, tenkantis 100 tūkst. gyventojų, pagal lytį, amžių ir išnaudojimo formą</font></div><font size='2'><font size='2'><font size='2'><font color='#ff0000'><b><span style='color:rgb(0, 104, 157);'>     Matavimo vienetai: </span></b></font></font>Asmenys</font><font size='2'><br /></font></font></div><div><font size='2'><font color='#ff0000' style='font-size:small;'><b><span style='color:rgb(0, 104, 157);'>Šaltinis:</span> </b></font><span style='font-size:small;'>Lietuvos Respublikos vidaus reikalų ministerija, </span>Valstybės duomenų agentūra</font></div><div><font size='2'><span style='font-size:small;'><br /></span></font></div><div><div><font size='2'>EN</font></div><div><span style='color:rgb(0, 104, 157);'><font size='2'><b>Goal 16. Promote </b></font></span><font color='#00689d' size='2'><b>peaceful and inclusive societies for sustainable development, provide access to justice for all and build effective, accountable and inclusive institutions at all levels:</b></font><div><font size='2'>Effectively implemented human rights and freedoms contribute to ensuring democratic values, strengthen security, democratic society, and reduce crime and discrimination. In order to achieve this goal, it is necessary to enable people to defend their rights by legal means, to ensure transparent and efficient performance of institutions. In Lithuania, it is aimed to create safe living environment, fight against serious and organized crimes and provide qualified assistance to victims.</font><div><font size='2'><font color='#ff0000'><b><span style='color:rgb(0, 104, 157);'>16.2. Goal:</span> </b></font>End abuse, exploitation, trafficking and all forms of violence against and torture of children.</font></div><div><font size='2'><font color='#ff0000'><b><span style='color:rgb(0, 104, 157);'>16.2.2. UN indicator:</span> </b></font>Number of victims of human trafficking per 100,000 population, by sex, age and form of exploitation.</font></div><font size='2'><font size='2'><font size='2'><font color='#ff0000'><b><span style='color:rgb(0, 104, 157);'>     Units: </span></b></font></font></font></font><span style='font-size:small;'>Persons</span></div><div><font size='2'><font color='#ff0000'><b><span style='color:rgb(0, 104, 157);'>Source:</span> </b></font>Ministry of Interior, </font><font size='2'>State Data Agency</font></div></div></div>
--b7d72677812d7812bcc692db225c8065
Content-Disposition: form-data; name="snippet"

16.2.2e. Victims of human trafficking for forced marriage
--b7d72677812d7812bcc692db225c8065
Content-Disposition: form-data; name="tags"

SDG,SDG16,DVR,DVR16,rodiklis 16.2.2e.,rodiklis 16.2.2.,indicator 16.2.2e.,indicator 16.2.2.,target 16.2.,uždavinys 16.2.,tikslas 16,goal 16,16.2.2e.,16.2.2.,16.2.,16,LT,LT16,Fl,FL16
--b7d72677812d7812bcc692db225c8065
Content-Disposition: form-data; name="culture"

lt
--b7d72677812d7812bcc692db225c8065
Content-Disposition: form-data; name="accessInformation"

Vidaus reikalų ministerija, Valstybės duomenų agentūra
--b7d72677812d7812bcc692db225c8065
Content-Disposition: form-data; name="licenseInfo"

<a href='https://creativecommons.org/licenses/by/4.0/' rel='nofollow ugc'><img alt='Creative Commons License' src='https://i.creativecommons.org/l/by/4.0/88x31.png' style='border-width:0;' /></a><br />This work is licensed under a <a href='https://creativecommons.org/licenses/by/4.0/' rel='nofollow ugc'>Creative Commons Attribution 4.0 International License</a>.
--b7d72677812d7812bcc692db225c8065
Content-Disposition: form-data; name="typeKeywords"

ArcGIS Server,Data,Feature Access,Feature Service,Service,Singlelayer,source-2ff85137a9684a3bb6bdfc206cae2162
--b7d72677812d7812bcc692db225c8065
Content-Disposition: form-data; name="extent"

[[20.9542, 53.8967], [26.8356, 56.4504]]
--b7d72677812d7812bcc692db225c8065
Content-Disposition: form-data; name="url"

https://osp-sdg.stat.gov.lt/arcgis/rest/services/GOAL16/LT_16_2_2e/FeatureServer
--b7d72677812d7812bcc692db225c8065
Content-Disposition: form-data; name="text"

null
--b7d72677812d7812bcc692db225c8065
Content-Disposition: form-data; name="thumbnail"; filename="ago_downloaded.png"
Content-Type: image/png

 PNG
..
--b7d72677812d7812bcc692db225c8065--

Which gives response:

HTTP/1.1 200 OK
Date: Wed, 30 Oct 2024 14:18:45 GMT
Content-Type: application/json;charset=utf-8
Content-Length: 107
Connection: keep-alive
Vary: Origin
X-Content-Type-Options: nosniff
Cache-Control: no-cache
Pragma: no-cache
Expires: -1
Strict-Transport-Security: max-age=31536000
Vary: Accept-Encoding

{"error":{"code":400,"messageCode":"CONT_0057","message":"Invalid JSON in 'text' parameter.","details":[]}}

Expected behavior

Expected cloned item appear in target and cloned item references same external feature service / feature layer.

Platform (please complete the following information):

  • OS: Linux
  • Browser: -
  • Python API Version: 2.4.0

Additional context

We are doing migration of ArcGIS Online content from one account to another. Cloning is being done in bulk. Some items causes the problems. They exists in source, but can't be cloned to target.
@rkraujutis rkraujutis added the bug label Oct 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@rkraujutis