400 Bad Request when uploading to server

[accraze]

I am trying to run pypicloud on an EC2 instance. The UI works fine and I can log in, upload and fetch there just fine, however, I can't seem to search or upload packages.

For example, when I try to upload a package like:

twine upload --skip-existing dist/* -r pypicloud

I get HTTPError: 400 Client Error: Bad Request for url: http://<domainname>/pypi

I have already configured my .pypirc file as mentioned here and confirmed that I get a 401 unauthorized error if I try to upload a package with a non-existent user. I have set pypi.default_write = authenticated and am also logging in using admin credentials. I have also tried the /pypi and /simple endpoints but I still keep getting the 400 error.

Any ideas?

Here is my server.ini

[app:main]
use = egg:pypicloud

pyramid.reload_templates = False
pyramid.debug_authorization = false
pyramid.debug_notfound = false
pyramid.debug_routematch = false
pyramid.default_locale_name = en

pypi.default_read =
    authenticated
pypi.default_write =
    authenticated

pypi.storage = s3
storage.access_key = *********
storage.secret_key = *********
storage.bucket = *********


db.url = sqlite:///%(here)s/db.sqlite

user.admin = *********

auth.admins =
  admin

# For beaker
session.encrypt_key = RANDOM_STRING
session.validate_key = RANDOM_STRING
session.secure = False

###
# wsgi server configuration
###

[uwsgi]
paste = config:%p
paste-logger = %p
master = true
processes = 8
reload-mercy = 15
worker-reload-mercy = 15
max-requests = 1000
enable-threads = truesocket = 127.0.0.1:6543
virtualenv = /home/pypicloud/venv


###
# logging configuration
# http://docs.pylonsproject.org/projects/pyramid/en/latest/narr/logging.html
###

[loggers]
keys = root, boto, pypicloud

[handlers]
keys = console

[formatters]
keys = generic

[logger_root]
level = INFO
handlers = console

[logger_pypicloud]
level = DEBUG
qualname = pypicloud
handlers =

[logger_boto]
level = WARN
qualname = boto
handlers =

[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic

[formatter_generic]
format = %(levelname)s %(asctime)s [%(name)s] %(message)s

[stevearc]

I tried running twine upload on a server and it appears to complete normally. I'd recommend trying an upload with python setup.py sdist upload -r pypicloud to see if it's something with twine or if it's related to the configuration or server. The other recommendation is to check the server logs and see if they have any more detailed information about the error or where it's coming from. As it is, it's kinda hard to tell what part of the request was bad. I believe that pypicloud should include some additional information in the response, but either it's not or twine isn't printing it out (another reason to try setup.py)

[illagrenan]

I have similar problem when trying to upload *.whl file. Twine fails on this error:

root@964acbd92db9:/# twine upload --verbose /dist/* --skip-existing --username *** --password *** --repository-url=https://***/pypi/
Uploading distributions to https://***/pypi/
Uploading fasttext-0.8.22-cp36-cp36m-linux_x86_64.whl
 25%|██████████████████████████████████████████████████████████▎                                                                                                                                                                              | 576k/2.25M [00:56<02:48, 10.5kB/s]Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 354, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.6/http/client.py", line 1065, in _send_output
    self.send(chunk)
  File "/usr/local/lib/python3.6/http/client.py", line 986, in send
    self.sock.sendall(data)
  File "/usr/local/lib/python3.6/ssl.py", line 972, in sendall
    v = self.send(byte_view[count:])
  File "/usr/local/lib/python3.6/ssl.py", line 941, in send
    return self._sslobj.write(data)
  File "/usr/local/lib/python3.6/ssl.py", line 642, in write
    return self._sslobj.write(data)
BrokenPipeError: [Errno 32] Broken pipe

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 638, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/usr/local/lib/python3.6/site-packages/urllib3/util/retry.py", line 367, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.6/site-packages/urllib3/packages/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 354, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.6/http/client.py", line 1065, in _send_output
    self.send(chunk)
  File "/usr/local/lib/python3.6/http/client.py", line 986, in send
    self.sock.sendall(data)
  File "/usr/local/lib/python3.6/ssl.py", line 972, in sendall
    v = self.send(byte_view[count:])
  File "/usr/local/lib/python3.6/ssl.py", line 941, in send
    return self._sslobj.write(data)
  File "/usr/local/lib/python3.6/ssl.py", line 642, in write
    return self._sslobj.write(data)
urllib3.exceptions.ProtocolError: ('Connection aborted.', BrokenPipeError(32, 'Broken pipe'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/twine", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.6/site-packages/twine/__main__.py", line 28, in main
    return dispatch(sys.argv[1:])
  File "/usr/local/lib/python3.6/site-packages/twine/cli.py", line 75, in dispatch
    return main(args.args)
  File "/usr/local/lib/python3.6/site-packages/twine/commands/upload.py", line 125, in main
    return upload(upload_settings, args.dists)
  File "/usr/local/lib/python3.6/site-packages/twine/commands/upload.py", line 86, in upload
    resp = repository.upload(package)
  File "/usr/local/lib/python3.6/site-packages/twine/repository.py", line 160, in upload
    resp = self._upload(package)
  File "/usr/local/lib/python3.6/site-packages/twine/repository.py", line 152, in _upload
    headers={'Content-Type': monitor.content_type},
  File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 581, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 498, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', BrokenPipeError(32, 'Broken pipe'))

I was trying to upload Python build of this project: https://github.com/facebookresearch/fastText. I think this is not a problem with Twine because I cannot upload *.whl (this is the file: fasttext-0.8.22-cp36-cp36m-linux_x86_64.zip) directly with GUI.

[stevearc]

I tried uploading that file and it seems to work fine for me with S3 on the backend. My guess is there may be some sort of connection issue? Check the server logs and see if they have any more information. From the twine client it seems like the TCP connection is being broken during the upload process, but there's no clue as to what may be happening on the server side that causes that.